Seasoned project managers will tell that delivery of software is result of many trade offs. The main trade off is between Time (when project will finish) and Scope (how much will be done). This post will show that using right tools you could gain improvement for both variables.

While it is possible to create orderly step-by-step process for increasing productivity of software teams, it will never be ideal – too many variations and situations will hinder it usefulness. I believe in set of useful tool that could be combined to craft custom optimal solution.

Strategies

There are several strategies that lead to increase in productivity – how many units of scope software team can produce within fixed time.

1. Increase Capacity (Capacity) - increase capacity by hiring more people or increasing work hours
   • productivity is increased as result of more resources and hours available
2. Improve Value Stream (Value) - increase added business value on each step and reduce waste and overhead
   • productivity is increased as result of optimized delivery of value
3. Adaptat to Reality  (Adaptation)- learn from practice and mistakes, validation of ideas by reality, adapt to changing situation
   • productivity increased as result of early corrections and improving how things are done
4. Empower Individuals (Individuals) - boost people knowledge, skills, morale and focus
   • productivity is increased as result of higher individual performance and motivation
5. Enhance Communication (Communication) - improve communication and mutual understanding inside and outside of the team
   • productivity is increased as result of availability of necessary information, clarity of what should be done and exchange of ideas for implementation
6. Organize Better (Organization) –  structure team and assign roles for better coordination and decision making
   • productivity is increased as result of better decisions and focus on important areas
7. Expand Expertise (Expertise) - increase range of skills and services offered by team
   • productivity is increased as result of better execution of necessary project activities
8. Scale Externally (Externality) – outsourcing and involvement of external communities
   • productivity is increased as result of involvement of more people outside of team
9. Tame Complexity (Design) – manage complexity and provide simple and well designed solutions
   • productivity is increased as result of reducing complexity burden on software development
10. Preserve Quality (Quality) – use defensive tactics to ensure high quality
    • productivity is increased as result of preventing system flaws and reduced effort to fix bugs

• People-oriented – people are the creators of software and have major effect on output
• Process-oriented – the way how people work has significant impact on outcome
• Development-oriented – development practices and approach to the system implementation matters a lot

People-oriented tools

Strategies: Capacity , Expertise , Communication ,  Organization

New people increase volume of possible work within the same time and new expertise increases range and quality of tasks. However, the increase in output is not linear and quickly diminishes, because more people cause communication and coordination overhead. In addition, compensation increase make this tool expensive and less attractive.

Resources:

• Methodology and Team Size, Alistair Cockburn
• Team Size Matters, schmula

2. Motivate

Strategies: Individuals , Quality

Happy motivated people are more productive, focused and concerned about quality and end result. While there is no common receipt for each individual, I believe there are some strong motivators: interesting and meaningful work, fair compensation, control over own tasks and outcomes, ability to learn and professionally grow, comfortable workplace and adequate tools, empathetical and caring management and so on.
Daniel H. Pink in Drive thinks that people are motivated by three elements:

• Autonomy: People want to have control over their work.
• Mastery: People want to get better at what they do.
• Purpose: People want to be part of something that is bigger than they are.

• Professional Growth

Tips: Motivate individually, but set also group targets to encourage cooperation

• Drive, Daniel H. Pink
• The Ideal Software Company. Utopia?

3. Professional Growth

Strategies: Individuals , Expertise , Quality , Adaptation

Software professionals should learn a lot to become productive and proficient. People learn in several ways and learning from mistakes is the most memorable, but expensive way. Learning by self-study, formal training and coaching by experienced colleagues are more effective ways. Encourage and direct learning to grow high performance experts and team. And certainly, continue learning as much as possible from mistakes and practice. :)

• How to Become an Expert. Top 7 Qualities
• How to Become an Expert. The Effective Way

4. Overtime

Strategies: Capacity , Individuals , Quality

Nobody recommends involuntary overtime for a long time as benefits will disappear quickly when a stressed team lowers quality and starts breaking apart.

5. Right leaders

Strategies: Communication , Organization

Good leaders jell teams, align people with goals, boost energy and remove barriers. Leaders vary from commanders to visionaries and each style has merits under specific circumstances. Right leaders are essential for the project success.

Tips: Adjust leadership style to situation

• What is The Best Leadership Style for The Software Team?

Strategies: Expertise , Organization

A software team requires range of skills and involved in different activities to achieve end goal – implemented software system for customer needs. Beyond software implementation the team deals with a customer domain, company vision, market demands, technologies, operation environments and other project aspects. The team is involved in research, analysis, coordination, design, architecture, usability, testing, deployment, hosting and other activities. Therefore team players should be able to play different roles and have expertise to cover various aspects to ensure good end results.

Resources:

• Why specialization in Software development is bad for business

Strategies: Capacity , Expertise , Externality , Organization , Communication

Tasks and challenges should match skills and experience of team players. It doesn’t make sense for backend developers to convert Photoshop files into html or conduct marketing research and focus groups. If team cannot afford to hire a permanent specialist, they should try to find professional mercenaries who will do work faster on higher quality level.

Strategies: Individuals

Flow (or Immersion) is a state of mental focus so intense that awareness of the real world is lost, generally resulting in a feeling of joy and satisfaction. When learning and tasks are too easy, people become bored. If they are too complex they become stressed and frustrated. Flow happens when perception and understanding are challenged near capacity without being exceeded.

• ability to focus
• clear goals
• immediate feedback
• control over actions, activities and the environment

• Flow: The Psychology of Optimal Experience, Mihaly Csikszentmihalyi

Strategies: Quality , Organization , Individuals

Few people will be always disciplined no matter what and few will always violate rules. Majority are context sensitive – relax than discipline is low and work hard when discipline is enforced. Build a minimal set of rules approved by majority and stick to these rules.

Resources:

• Lost Personalities: How our company alters us

Process-oriented tools

10. Increase feedback and early practical use of ideas

Strategies: Adaptation , Quality , Value

Reality is complex, fluid and unclear. Many assumptions and plans quickly go bust. The process that has built-in mechanism for signaling problems and self-correction is the most effective. Rapid feedback is the core part of such process.

Tips: Iterative development is one of the best examples of the process that relies heavily on feedback.

11. Build alignment

Strategies: Communication , Organization

People jump significant gaps to understand each other – sometimes without much success. Clear vision, smart priorities and honest evaluation allow team players moving in unison and enforce each other. Good understanding focus people on right goals and reduce wasteful activities. Communication and trust are important for alignment.

• vision – project direction and top-level goals
• interests – mix of personal, customer and companies interests (sometimes conflicting)
• understanding – clarity about what others mean
• trust – confidence in other people intentions and promises

12. Integrated decision making

Strategies: Communication , Quality , Adaptation

See Also:

• Self-organization

As project progresses you will find problems, wrong assumptions and challenges. You will have better experience and understanding. Even brightest ideas, best practices and excellent analysis and design could become outdated and ineffective. Continuously look for improvements, eliminate waste and bottlenecks.

Resources:

• How to rescue failing software projects: The Toyota Way
• Theory of Constraints
• Kaizen

Strategies: Individuals , Quality , Value

Highly productive team shouldn’t waste time on routine or non-core activities. Routine work should be automated and non-core work outsourced or covered by external components. The team should focus on high value-added distinctive work for business and application domain – this will bring maximum business results with minimal development effort.

• system thinking
• understanding what really matters and what are limitations
• broad vision and out-of-the box thinking
• knowledge of environment, context and system outside relations
• synthesis of concepts, information and experience

Strategies: Capacity , Expertise , Externality , Organization

Strategies: Value , Communication

You can better understand and change scope of application if you actively work with customers and become partners

• establish ubiquitous language – for better understanding of ideas and translation them into code
• tap into experience and get better insight for business domain and processes
• transform ideas – simplify and find better alternatives

• reducing scope
• simplifying solutions
• finding good enough alternatives

• Pareto Principle
• Ubiquitous Language
• Software Development is The Flow of Ideas. The Rest is Secondary

18. Select Right Process Flow

Strategies: Organization , Value

Select most optimal process for creating value.

• real-time flow –  immediately push tasks for implementation (continuous development flow for highly organized co-located teams)
• pull systems (Kanban) – pull tasks when previous batch is finished and the team is ready to continue (iterations and backlog)
• schedule the workload (Heijunka) – create schedule for contributors with limited availability (outside consultants or shared specialists as designers)

Tips: Use different flows for distinct stages of software creation. For example, a team process can internally use real-time flow, with external teams – Kanban and with consultants Heijunka.

Resources:

• The Toyota Way and Development Process

19. Self-Organization
Strategies: Communication , Individuals , Adaptation , Value

Tips: Not every team can self-organize – it requires maturity. And loss of direct control could be a problem for management if a project departs from company strategy and desired parameters (as budget, standards, resources).

• What is The Best Leadership Style for The Software Team?
• Organizing Self-organizing Teams
• Agile Self-Organizing Teams

Development-oriented tools

20. Increase reuse

Strategies: Design , Quality , Adaptation

Reuse code  (components, libraries, solutions) as much as possible to speed up development and minimize code base. Reuse will bring familiar tested solutions and prevent frequent re-inventions of the wheel.

See Also:

Tips: Design for reuse requires skills and time. It is easy to overly complicate the system by pursuing reusable code. Evolve and refactor code to achieve practically good reusability. Sometimes custome solutions are lighter and better suited for specific purposes than generic and reusable.

21. Generalize and Simplify
Strategies: Design , Quality , Communication

Complexity and poor understanding of the system is one of the worst problems in development. Design, code and concepts should be as simple as possible and clearly understood by the team. Simplicity and clarity are difficult to achieve and require special effort. It is much easier to produce convoluted over-engineered solution for complex problem (Ball of Mud) than simple and elegant solution.

Tips: Great design achieve simplicity by abstracting numerous details into simple concepts in process of solution evolution and learning from implementation.

• Big Ball of Mud, Brian Foote
• 11 Laws of The System Thinking in Software Development
• The Elements of Pragmatic Programming Style. Approach.

Innovation is expensive, uncertain and leave you with custom solutions on your own for support and future development. Piggyback on actively involving products, especially for non-core problems where good and proven solutions are available.Tips: Innovate to gain business advantage or solve serious technical challenges if you have an outstanding team and well-defined problems.

Resources:

• Should An Effective Developer Innovate, Imitate or just Integrate?
• Selecting The Best Strategy for Software Teams: Retreat, Evolution or Revolution

23. Domain Specific Design

Strategies: Design , Communication

Keep technical ideas and code concepts close to the customer domain. You will achieve consistency, better sync between system and business concepts. You can easier translate customer ideas into the system and customers will understand your implementation better.Tips: Domain Driven Design is one the best approaches to achieve synthesis of technical and business ideas for complex domains.

Resources:

• Domain Driven Design, Eric Evans
• The Secret of Building Effective Software Systems
• The Elements of Pragmatic Programming Style. Composition.

24. Evolution

Strategies: Adaptation , Value , Design

Start with the simplest primitive solutions, enforce early use and seek for feedback. Adjust and improve design before continuing adding new features. Look for better ideas and abstractions. Delay fundamental design decisions until you are confident about system development direction and validated ideas on practice. Do not over-engineer and add unnecessary features.

Tips: Evolution main deficiency is slowness. Modern businesses often don’t have time for many improvement cycles – they want results now.

Resources:

• What can Software Development learn from Biological Evolution?
• Ideas in Software Development: Revolution vs. Evolution. Part 1.

• How a beautiful software system becomes Frankenstein

25. Open architecture and API

Strategies: Externality , Design

Open your solution for external developers who can extend and add more useful features. If you are lucky, you will get much more functionality than you can build internally while still controlling a core solution (iPhone is a perfect example).

Also Open API (convenient and supported by good documentation) give developers eligible ways to interact with you system without hacking or completely ignoring it.

Tips: Once Open API is used it becomes liability – next design decisions will be affected by compatibility and legacy concerns.
Resources:

• API

26. Follow Design Principles

Strategies: Design

Some of design principles

• You should be able to extend system without modifying it
• A class should have one, and only one, reason to change
• Depend on abstractions, not on concretions.
• Make fine grained interfaces that are client specific.

Famous design patterns are built around these principles. Follow sound design principles to build flexible, easy to expand system.

Resources:

• Principles of OOD, Rob Martin
• The Elements of Pragmatic Programming Style. Approach.
• The Elements of Pragmatic Programming Style. Composition.

27. Shared Code, Ideas and Standards

Strategies: Design , Expertise , Communication , Quality

The team should collectively own code and pursue consistency and use of best practices. Otherwise the system will become clash of isolated impassible code islands – scary for people who didn’t build them. Share code and learn from each other to build outstanding system with high quality and integrity of every component.

Tips: Pair Programming is one the best ways to share code, ideas and best practices
Resources:

• Collective Code Ownership
• Pair Programming

28. Use Better technology and tools

Strategies: Individuals , Design , Value

Tips: Obsession with new technologies and race to use them is dangerous for project. Keep sanity until new technology is proven to be beneficial :)

• http://www.sdtimes.com
• http://www.computerworld.com

Guide for increasing team productivity

Questions

A software project is a creative, unique and therefore unpredictable endeavor. We are not building the same thing over and over again, but solve new problems, address increasing demands and use perpetually changing technologies. Under these conditions, people – smart, creative and productive – are the most important factor of success . Software development process can only support and compliment these people, but it cannot guarantee success alone and make the factor of people negligible.

But, business wants predictable, reliable and successful results. I bet they don’t want to be at mercy how cards are shuffled in their talented development team. The answer is in establishing a process that increases chances of success and aligned with present nature of software development (unpredictable, empirical and heavily dependent on people).

The Toyota Way can be a great example that worth to learn. Toyota evolved from a small looming equipment shop to the largest car manufacturing company. The main foundation of successful growth is the system of few core principles that enables best quality, high productivity, lowest cost, shortest time and long-term success.

Agile and Lean Software Development adopted some practices from Toyota. Lean Software Development: An Agile Toolkit  (by Mary Poppendieck, Tom Poppendieck) is an excellent book that review Lean practices and principles in details. This post focuses more on philosophy and the system of practices as a whole that can turn software development team into a smart reliable machine.

1. Value

Why customer will pay money for your product? How does your company creates value?

All we are doing is looking at the time line from the moment the customer give us an order to the point when we collect the cash. And we are reducing that time line by removing the non-value-added wastes.

- Taiichi Ohno, founder of TPS

The only thing that adds value in software development is transformation of information and code into what customer wants. It is very important for software team to know how does it create value for customers. Or simply, decrypt the formula:
Customer (needs) -> Cash (delivered software)

Software Development Value Stream
Problems, needs and ideas are translated into units of development – functionality, user experience (interface, interactions, usability) and system qualities (performance, reliability, security, etc). A software development team transforms these pieces of requirements into a computer system using value-added and many not-so-value-added actions.

The ultimate goal of Toyota Way is to eliminate non-value-added activities while keeping the best quality of the product.

2. Flow

What is the process of creating value in your company?

Properties of the optimal flow for software development:

• cut back to zero the amount of time that any unit of work is sitting idle or waiting for somebody to work on it
• move design, code and information fast, link processes and people together that problems surface right away

One-piece flow
The most effective flow is one-piece flow – a customer need is immediately converted into a delivered software solution. In the same time this is the most demanding flow: a customer, designers, developers, testers and system administrators should be dedicated to the project, immediately available and work as one team together until the need is implemented in a live system.

One-piece flow has significant benefits (if it is implemented right):

• Quality - defects detected quickly; close proximity and connections ensure that necessary information is available immediately and important details are not missing; problems are solved quickly
• Flexibility - flexibility to respond and do what customer really wants in real time
• Productivity - the team engaged in high value added continuous work
• Morale - immediate results of work bring sense of accomplishment and job satisfaction
• Cost - lean production with minimal waste

However, in reality one-piece flow is difficult to achieve in many projects:

• a customer is not engaged all the time
• complex business domain requires analysis and preparation before development can start
• project activities involve people with different skills that are not available immediately
• deployment of mission-critical systems demands intensive testing and scheduling

• management, marketing, end-user expect specific time-lines, features and support

• and so on

Toyota uses two additional principles to overcome challenges of one-piece flow: pull systems and leveling out the workload.
However, Toyota Way says: “Flow where you can, pull where you must“. The big challenging goal is to turn eventually all the processes into one-piece flow.

Pull system (Kanban)
Kanban is a signaling system to trigger action. It adds to flow small buffers pulled by customer demand. Kanban allows optimal use of people and natural breaks in processes.

Agile iterations and Scrum sprints are close to Kanban ideas. A team pulls new user stories every time-boxed iteration from the backlog to design, implement, test and deploy within the same iteration.

However, it is not classical Kanban. An original idea would be to pull small batches of user stories for development after a team has finished a current batch (not at the end of fixed time iteration). In parallel, QA pulls implemented user stories for testing and so on. A customer can continue preparing next batches of work based on their priority of needs.

Main ideas

• provide dependent down stream teams with what they want, when they want and in amount they want
• just-in-time – started by high priority needs
• minimize work-in-progress and untested code
• frequently push what a customer can use right away
• be responsive to day-by-day shifts in demand instead of relaying on long-term schedules

Kanban is effective way to synchronize people and teams work without bringing them into continuous flow, where they should be always ready for the next unit of work.

Level Out the Workload (Heijunka)
The main idea of Heijunka is to level production by volume and activities mix.

• it is not built around actual flow of customer orders
• workload is leveled for the period considering previous history and scheduled projects

In software development workload could be leveled and scheduled when people could make reliable guess for what is expected. It is different from Kanban that works as just-in-time system adapting to the current workload.

Good candidates for Heijunka are

• standard procedures (automated testing, performance testing, deployment)
• recurring events (meetings, planning, demo, code reviews)

• scheduled tasks for people with limited availability (external consultants, customers, management)

• tasks with high confidence in estimation, usually done many times before (initial requirement exploration, design mockups, security audit, training new team members, reusing components and solutions)

Benefits:

• predictability and active planning

• balanced use of people, ability to schedule their involvement

• smoother demand on customers, shared resources and contractors

Toyota Way is focused on eliminating waste of non-value added work (Muda). Heijunka eliminates two additional types of waste:

• unevenness in project schedule (Mura)
• overburden of people and systems (Muri)

Building your own flow

These three approaches can help you to build most optimal flow. Consider cons and pros:

One-piece flow (push)
pros: minimal waste, fast implementation, low cost, quick problems resolution
cons: uneven workload, requires full availability and high level of engagement
when to use: for one collocated team work that is dedicated to one project

Kanban (pull)
pros: dynamic adaptation to load; synchronization of different teams and disconnected processes
cons: low predictability for completion; buffers hide waste
when to use: disconnected or involved in the multiple projects teams; activities that need people with limited availability

Heijunka (level)
pros: good predictability, ability to plan, balanced use of people: avoid overburden and uneven workload
cons: non-adaptive, inefficient for tasks with unpredictable effort (a lot in traditional software development)
when to use: activities that are standardized, highly predictable or have to be scheduled

You cannot build reliable and optimal flow from the scratch based on a theory only. The best process for your projects will emerge as result of evolution, problem solving and eliminating waste. This is a topic of the next post.

If some problem occurs in one-piece flow manufacturing then the whole production line stops. In this sense it is a very bad system of manufacturing. But when production stops everyone is forced to solve the problem immediately. So team members have to think, and through thinking team members grow and become better team members and people.

- Teruyuki Minoura

References:
The Toyota Way by Jeffrey Liker
Lean Software Development: An Agile Toolkit  by Mary Poppendieck, Tom Poppendieck

People are amazing, surprising and interesting. They change reality with power of thought and make things happen. What is most exciting – all people are completely different in their attitudes and behavior. But this comes with price – it is difficult to understand people and even more difficult to find the best way to deal with them.
Many people, who see programmers as extensions of their computer systems, will be surprised to discover that programmers are amazing individuals too. Programmers exhibit similar to other people behavior, they have different personalities and need individual approach.

I offer in this post a simple theory about Three Dimensions of a Software Programmer that could help to put relations with these individuals on some rational basis.

Axioms

There are two basic axioms in foundation of the theory

1. Constancy - some programmers consistently outperform others under same conditions.
2. Variability - performance of a programmer varies under different conditions.

Talents (constants)

Talents are recurring patterns of thoughts, feelings and behavior that can be productively applied. – First, Break All the Rules
Genes and upbringing form an adult human with specific talents and personality, strengths and weaknesses. These core individual characteristics of a programmer are difficult to change (except some limited success with consumption of large quantities of beer). Programmers are different – creative, thorough, funny, systematic, laid-back, curious, analytical, spontaneous and so on. Some people cannot be programmers at all.
Each unique programmer has a base performance level that is constant and almost impossible to change.

Three Dimensions (variables)

Now we can move to the part of equation that can be changed. Can we make programmers permanently performing above their base level? To answer this question lets review three variables of programmer’s performance.

X. Energy
measures amount of work and power of ideas that could change and improve reality.
Here I consider a specific kind of energy – creative positive energy that is directed to build better software and aligned with company and customers goals. Individual’s desire to act, create and achieve is the source of this energy. As we discussed above, part of this energy is based on character and predetermined. However, there is a big variable component of individual energy that is driven by motivation and environment.

Y. Discipline
measures ability to focus and follow necessary steps to achieve goals with good quality.
Accountability, self-organization and focus are core elements that keep individuals to perform on a high level. Personality matters here – some people will promise and forget immediately, give up experiencing first difficulties or become distracted by more interesting tasks. Some will fight until the end and go beyond limits to meet commitments, exceed expectations and bring excellent results. However, significant part of programmer’s discipline is variable and depends on context and expectations.

Z. Expertise
measures knowledge and experience in programming, technology and a customer domain.
Expertise includes know how to build software, make right decisions and implement good solutions with minimal troubles. There is no doubt that expertise is growing with time, but individual variations are big. Many people consider expertise as the only important characteristic of a programmer. I believe that it has much lower value without two other variables.

Programmer’s Archetypes

FfHII

How to Improve

A programmer can be 10 times more productive than other programmer and 10 times more productive than himself under different conditions. Therefore right people under right conditions could be 100 times more productive than others.

How can you influence programmer’s performance with these three dimensions? Hopefully, you found right people with right talents. It is a waste of time to change personality and push in direction where talent is absent. However, it is possible to amplify natural strengths and create conditions for super productivity.

Energy
goal: make people empowered, motivated, interested and satisfied; engage them to perform above their base level of energy

job design

• interesting work – brings inspiration and motivation
• control – over own work and decisions – pushes people forward
• self-organization – opportunities to find own ways that bring best results
• creative tension – goals on the edge of capabilities to focus energy and open new energy sources
• defined outcomes – clear direction that reduces anxiety and doubt and increases productivity

psychology

• flow – put mind in a state of energized focus, full involvement, and success
• positive experience – good emotions and healthy team relations; comfortable and trusted environment
• personal interests – decision making considers and takes care of individual interests
• right incentives – compensation, rewards, recognition are the great source of energy and motivation

environment

• space for mistakes – tolerance and open ways to fix mistakes make people think about moving forward instead of covering assess
• productive environment – where people think about work instead of nuisances
• fit – between personal and company goals, culture and views – amplify people desire to work
• reasonable pressure – normal workload, minimal stress, time slack to avoid energy drains

Discipline
goal: make people focused, responsible and aligned; make them performing in the right way

support system

• established process – align actions with goals and support their execution (Agile is the best way to go)
• transparency – reasons for decisions should be clear, problems should surface quickly
• pull system (kanban) – signaling system to trigger action based on actual needs
• control mechanisms, check and balances, audit, risk management (e.g. iterations)

zone of responsibility

• clear areas of responsibility – what is included and what is not
• balance of responsibility and authority – adequate authority to accomplish goals; restrictions to avoid damage
• rules of engagements – when, where, and how force (bold actions) shall be used
• match confidence with responsibilities – an individual is prepared and ready for the new responsibilities

standards

• safety regulations – mandatory rules for everyone
• professional standards – recommended parameters of work and products (security, performance, availability, etc.)
• best practices – spread approaches that work

Expertise
goal: make people knowledgeable, experienced and growing based on their talents

career

• right career path – built on persons strengths – enable growth in area of competence and natural strengths without switching into paths that don’t match talents (e.g. management). Avoid consequences of raising to the level of incompetence (Peter Principle).
• broadbanding – build levels of achievements and pay schemes that allow to achieve and earn more within the same profession
• match expertise and challenges- keep people always interested and challenged, avoid boring or over complicated tasks
• assist self-discovery – help search for the better career and role

perspective

• diversity in tasks and activities – accelerates growth and understanding
• big picture, direct communication with customer – informed people make more optimal decisions
• exposure to different subject areas – broader knowledge opens new possibilities and solutions outside the narrow specialization

learning and practice

• feedback loop – continuous evaluation and improvements based on feedback
• opportunities for learning – grow expertise beside direct work tasks
• sharing, coaching, communication – peer and experts access and support for growth

There is no standard way to increase performance of an individual programmer. Each individual is unique and requires personal approach. The theory of Three Dimensions of a Software Programmer suggests framework for these approaches.
It takes a huge effort to understand every programmer in your team individually and craft personalized approach, path and conditions. But it makes sense to do. People will be productive, engaged and happy. The company will grow and succeed.

What could be a better outcome in our civilized hard working world?

Jim Highsmith says:

“I’ve been thinking recently that the term “self-organizing” has outlived its usefulness in the agile community and needs to be replaced. While self-organizing is a good term, it has, unfortunately, become confused with anarchy in the minds of many. Why has this occurred? Because there is a contingent within the agile community that is fundamentally anarchist at heart and it has latched onto the term self-organizing because it sounds better than anarchy.”

Army experts say:

“A culture of adaptability is one that accepts a lack of absolute control over events on and off the battlefield. Implementation requires revisiting mission orders or trust tactics. It necessitates raising the bar in the education, training and coaching of leaders and soldiers. It seems trite to suggest that an adaptive institution will reward those who, when the need arises, act without waiting for orders, but this also necessitates a climate that is supportive of those who act and fail to achieve stellar results. Instead of seeking perfection or optimum solutions, operators will find a solution that works locally and then exploit those results as a continual evolution facilitated by an organization adept at receiving and communicating such information.”

The modern Army faces challenges that demand from soldiers quick decision making on spot and reaction to unpredictable situations while keeping ethical reasoning and holding to the Army goals and values. Don Vandergriff and George Reed are confident that only small adaptive and learning units could handle such tasks – large, bureaucratic structures, with rigid lines of authority, are inherently slow to respond and adapt.

Jim Highsmith is a prominent advocate of adaptability, his book Agile Project Management is one of my favorite. But rejecting concept of self-organization he dismisses the essence of adaptive and learning agile teams. Certainly, self organization is not for every team, project or company, but removing the concept from Agile world is too radical step. Achieving the state of self organization should be a goal for any software development team – it is the most effective leadership style. Suggested Light Touch Leadership is not the best style, but only a stage in achieving true self-organization.

Why self organization is the best style? There are two main perspectives: theory of complex systems and human psychology.
Theory of complex systems shows that self-organized systems:

• better in selecting optimal state and local decisions
• robust
• effectively adapt to environment and use feedback loops
• often come up with emergent and unexpected solutions
• self-regulated and better cope with perturbations

Human psychology aspect adds that self-organized teams:

• more responsible for end results, self-disciplined and self-driven
• avoid dependency on the formal leader qualities
• motivated, initiative and willing to act
• enjoy work more
• better insured against groupthink, conformity and diffusion of responsibility
• not shifting judgment and decisions to others, better in finding alternative and balancing options
• every member is in charge, ready to step in as a leader and have incentive to develop leadership skills

A self-organized team is possible when people carry shared purpose, principles and values. They support and respect each other. And they want to succeed.

Jim, don’t take away from us concept of self organization. It is not anarchy, but the highest form of the software team organization, which I wish every software team could experience and achieve.

Resources:
No More Self-Organizing Teams, Jim Highsmith
No More Self-Organizing Teams. Not. An open letter to Jim Highsmith from Tobias
Old Dogs and New Tricks: Setting the Tone For Adaptability, Don Vandergriff and George Reed

Programmer’s Side

Programmers in the software team work well together, trust and support each other. But they don’t talk about their salaries – the most important reason why they work for the company. It is taboo. Tell them that one of their fellow programmers earn 50% more than others. You’ll see smiles disappear, people don’t look at each other and become silent. They will rebound, but these things leave scars on team morale and cohesiveness. Feel of unfairness is not the best companion in the software development.

But why does compensation is so important and sensitive for us?

• We work to live. We spend more than half of our conscious adult life earning money on work. We need money to live. Money enable for us almost everything material in this world. Any non-material incentives cannot compensate insufficient money income.
• Money are social status. The better compensation means you worth more and more attractive from social and even biological perspective. People strive for relative prosperity. They could accept less themselves than see a rival get more.

Company’s Side

The software company is interested in the best productivity, high quality and most optimal software solutions. That usually translates into lower cost, higher profit and happier customers.
How a company can make it possible? People are the most important factor to achieve these goals. Company just need to

• Keep programmers motivated, interested and satisfied. Programmers will be happy if they perceive compensation as fair, work as interesting and environment as comfortable.
• Reward desired behavior for the company success – accountability, creativity, intelligence, self-discipline, learning and may other useful qualities. Compensation should link these qualities to the individual, team and company performance.
• Attract and retain the best contributors and talents. Programmers should have opportunities for earning more for better contribution. Many of them possess entrepreneurial spirit and stiff salary ranges and limits for growth can make them leave.

What is the best way for the company to approach compensation and support these goals?

Compensation Challenges

A company should solve few challenges in order to build an effective compensation system.

• How can the company objectively evaluate individual accountability, creativity, intelligence, self-discipline, learning capabilities, experience, influence, coaching, attentiveness to details, vision, strategic thinking, cooperation, enthusiasm, work ethic, quality, speed, thoughtfulness, inspiring and motivation of other people and many other qualities? What of them are most important for the company success?
• How can compensation encourage collaboration, cement relations and avoid competition between team members?
• How to get great team productivity results, without negative impact on other teams and without sacrificing long-term perspective?
• Finally, what compensation system will be perceived as fair?

Considerations:

1. Objective behavior evaluation is not possible. How can evaluation account for the huge number of behavior variables? It is almost impossible. In addition, some people traits play well only if they are complimented by other people traits: visionary needs detail oriented performer, creative person needs pragmatic opponent, innovator needs patient adapter. That is why all traditional evaluations will never give true picture of contribution and certainly will miss important points.
2. Fairness is in transparent and equal criteria. There are different types of criteria possible: years worked in company, level of education and even produced lines of code :). However these criteria don’t guarantee desired results.
3. The most effective economic criteria – profit. The most effective and simple criteria is found on the market where company operates – profit. It is a bottom line and result of the whole company activity. This is a reason why company hires employees and their compensation should reflect company economic results.
4. Collaborate inside. Free market promote competition which could easily reach all levels in the company. Competition is good in a healthy dose, but too much can damage cooperative nature of the software development inside teams. Also competition between teams and business units is damaging too as it could cause suboptimal decision on the company level.
5. Compete outside. The solution is to make people compete with the same competitors as the whole company facing on market. Link compensation to the whole company economic performance. Measure Up.

Fair Compensation System

There is a wide range of existing compensation systems:

1. Closed vs. Open. Opens salaries will force compensation system to become fair. Secrecy is damaging and corrupting. Open salaries should eliminate personal bias, favoritism and undeserved power from management. People with higher salaries will feel pressure to justify them and show their best performance; otherwise their status will be questioned.
2. Fixed vs. Performance based. Fixed rates promote inefficiency and detachment from the company results. People should have direct dependency between results of their work and the company performance.
3. Driven by management or teams. Team members are in the much better position to determine most valuable contributors and ‘free riders’. Managers don’t have this insight into each team member work. They are not the best people to decide on individual contribution and compensation.
4. Equal salary for levels vs. Individual negotiations. While individual negotiations are more flexible, they could promote bargaining and lack of transparency. People should be more concerned about performance of the team and company and less about individual negotiations. Some compensation parameters could be negotiable, but they should stay within baseline for the same contribution level.
5. Non-money incentives and free perks. They are nice, but not important for all as oppose to compensation packages.

There is no one approach that works for all, but 3 approaches below could increase efficiency of the compensation system. Additional note: effective software team includes all contributors – business experts, designers, programmers, testers, system administrators, etc.

1. Compensation Composition. A company should be flexible in packaging compensation. Some people expect secure and stable income, some are willing to work harder and risk for opportunity to earn more. Use mutual funds approach – allow custom packages (Stability, Income, Growth) with various components. They could include base salary, profit sharing (both depending on teams) and monetary options (partly covered by the company): training, health insurance, pension plans, vacation, stocks, equipment, car expenses, etc.
2. Team Contributors. Team should have minimal number of contributor’s levels: Junior, Full and Senior Contributors. Most people in the team should be Full Contributors and only outstanding members should be Senior Contributors. Each level has fixed ratio of income (e.g. Full Contributor – 25% and Senior – 50% of Junior compensation). The same level makes roughly the same money (depending on packaging). The team makes decision about moving person from one level to another or removing from team at all.
3. Company input. Company provides initial number of roles, budget and profit share structure. Team contribution is measured based on the profit or other criteria if the company on early market expansion (number of users / traffic, share in cost reduction, etc.). Each team receives part of the profit from their activity. Other part comes from the company pool (formed from remaining profit of other teams). Therefore, teams are interested in both their and overall company performance. Product teams have larger share of their profit, infrastructure teams have larger share from the company pool.

Fixed secretive compensation turn a team into strangers or even rivals, concerned only about individual achievements . Open fair and performance-based compensation motivates, retains and attracts programmers eager to create great, useful and profitable software.

Interesting resources:

• Unjust Deserts, Mary Poppendieck
• Measure Up, Mary Poppendieck
• Extreme Programming Group discussion
• Why secret salaries are a baaaaaad idea, Alexander Kjerulf
• Fog Creek Compensation, Joel Spolsky
• Employee Rewards, Accel Team

What should be the qualities of the best leader for the software team: strong decisive knowledgeable or quiet supportive cooperative? Best leaders have two main concerns – people and getting things done (Blake and Mouton’s Managerial Grid). Both concerns have cumulative effect – high concern for people makes them motivated and therefore more productive, high concern for production creates sense of achievement and makes people more satisfied at work.

However, there is no one leadership style that suits all situations, projects and individuals. There are 4 main leadership styles (based on The Hersey and Blanchard model and Vroom and Yetton’s Normative Model), which could be effectively used in software development: The Commander, The Coach, The Supporter and Self-Organization.

1. The Commander

In tough times people need a strong leader – the person who can take responsibility and make decisions, the person who could motivate and inspire, the person who could move against winds and resistance.
Commanders are needed

• in crisis, when bold, quick and direct decisions should be made. The project is doomed, customers are leaving or there is fire in the building.
• when changes are necessary and there is strong resistance and barriers to make them. The leader creates a vision, finds a way to accomplish it and leads a change. The company moves to the new market or adopts new approaches as Agile.
• when people lack commitment, faith in company and willingness to work, because of previous failures, degrading environment or unpopular management. The leader can wake them up, inspire and move forward.

Commanders make and influence most decisions. The downside of this leadership style is that leader can slide to autocratic methods, demotivate and annoy people. Often decisions are not optimal – they don’t take in consideration all available information, options and perspectives. This style is effective in short-term, in long-term it could be dangerous for people and projects.

Skills and traits of Commanders are very important as they are the main factor of success. The Commander should be energetic, confident, decisive. Also as Commanders make most decisions they should be knowledgeable, forward-thinking and broad-minded.

2. The Coach

The Coach is experienced, knowledgeable and supportive person, who is needed when team lacks focus, expertise and understanding what should be done and how.
Coaches are concerned about growing people, creating enabling and trusting environment. This leader makes decisions collectively with a team. The Coach explains rationale behind decisions, listens and provides feedback. He encourages personal growth, builds long-term capabilities and prepares team and individuals for independent work.
This style has high relationship focus. The downside of this style could be micro-management and low concern for productivity.

Important qualities for Coaches: knowledgeable, empathetic and cooperative.

3. The Supporter

pignews

Supporters are needed to help teams, remove barriers and coordinate activities. The Supporter is an ego-less quiet leader and facilitator. Supporter makes joint decision with the team as equals, delegating majority of decisions to the team. In addition, this leader concerns about creation of harmony and balance between team members.
Important qualities for The Supporter: good communicator and facilitator, socially skilled and diplomatic

4. Self Organization

A cohesive, motivated and confident Team doesn’t need formal leaders. The team makes most decisions, while every member could step in and become leader in specific areas and situations. People are highly capable, committed and self-driven. The team should have diverse and independently thinking team members to prevent groupthink (see Wisdom of Crowds).

It is most preferable and most difficult to achieve style. Usually the team transcends through previous steps and becomes truly self-organized team after experiencing victories and failures, growing and gaining experience together. The Team have shared purpose, principles and values. Team members trust, support and respect each other. Almost everything is possible for such team.

Selecting Style

Teams and leaders use these 4 styles depending on a situation and specific people. The best leaders guide their team and people through these stages to the point when formal leaders are no longer needed – self-organizing team. It is possible that a self-organizing team will need a strong leader in tough time, a coach when moving to the new domain or a supporter to resolve external problems. However, the team, which experienced flow, independence and self-organization will strive to get back to this state again.

Important factors

1. Commitment - are people ready, motivated and willing to accomplish goals?
2. Competence and clarity of direction: do people know what to do and how to do it?
3. Cooperation and cohesiveness: can team effectively work together, make decisions and avoid conflicts? The larger group need more coordination.
4. Resources and support: does team have necessary tools, money, people, etc? Do they face serious barriers and require support?
5. External coordination: does team need to collaborate with other groups?
6. Leader - last, but not least: does leader have necessary authority, knowledge and experience?

All these styles could be effectively used in software development. But at the end, the most important factor of success is people. The style should match people commitment and capabilities. It is mistake to command skilled and confident team and it is mistake to give reins to unmotivated and inexperienced people. Leadership style is one of the secret ingredients of the Software Creation Mystery – it could bring great success or miserable failure on the same project with the same people.

Management consultant Tim Bryce doesn’t like programmers. Many programmers don’t like him either (here, here, here, here and probably in many other places).
Mr. Bryce’s view of programmers:

• programmers often bamboozle others and heighten their own self-importance
• the average programmer has a lower IQ than any other worker with a college degree
• programmers show signs of sloppiness and mental laziness
• they appear disorganized to make it difficult to judge how they are progressing on their work effort and reveal inadequacies in workmanship.
• the typical programmer often laments he/she is being overworked, underpaid, and unappreciated.
• to the programmer’s credit, they usually possess a curiosity about technological developments. However, this must be carefully nurtured by management – too much information may distract programmers from their job.

Responses on Mr. Bryce’s post (referenced above) provide many excellent points why Mr. Bryce is wrong. However, I want to comment points where he is probably right. Also I want to understand why some management consultants with more than 30 years of experience could have such view? Certainly, I reject Freudian view that an unknown programmer hurt feelings of Mr.Bryce in childhood (the main reason is that at this time the world had only few programmers and all of them are known).

Mr.Bryce’s target audience is not programmers (whose low IQ would probably prevent understanding his Theory P anyway), but IT managers and business decision makers. The underlying premise of Theory P is: “The more effectively we manage the people who program the computer, the better we can utilize the systems to support the information needs of the business”. This theory is not about live people, but about pragmatic business and lets consider the theory from this perspective. There are three points where Mr. Bryce could be partially right.

1. Programmers are not “Software Engineers”, but Translators.

I agree with this statement. Programmers indeed translate. However, I don’t agree with what they translate. Mr. Bryce writes that programmers translate “human understandable specifications into machine understandable instructions”. In reality, programmers translate fuzzy human needs and ideas, which are often far from not well-defined specifications. Sometimes the difference could be as between Picasso translating human feelings in art and house painter translating his customer orders into colored wall.
If the flow of ideas between customer and programmer is bad, translation will be bad, software will be bad and, therefore, programmers will look bad. And Mr. Bryce will be right (unless he starts using The System Thinking).

2. Many programmers are not “System Analysts”

Unfortunately, in many cases, it is true. Mr. Bryce writes: “A true “Systems Engineer/Analyst” studies business requirements, defines and develops business processes to implement the requirements, and specifies software requirements for programmers to implement. Regrettably, there are too few people performing this vital task and, consequently, the responsibility defaults to the programmer who is not necessarily equipped with the proper skills to perform the work”.
I do know many programmers who are not very good in understanding business requirements or don’t want to understand them at all. However, unlike Mr. Bryce, I believe that programmers must learn speaking business language, communicate directly with customers and understand their needs. I believe that true programmer’s job is not limited to writing instructions for computers. The professional programmers should become master translators of complex mental concepts into software.
Otherwise wrong interpretation of business concepts will make software bad and, therefore, programmers will look bad. And Mr. Bryce will be right again.

3. Programmers tend to fashionable, but not practical solutions

Mr. Bryce writes: “An elegant solution to the wrong problem solves nothing. It is important for programmers to learn to justify their technical recommendations from a business perspective.”
Indeed, many programmers are more interested in technical and even fashionable solutions without considering business perspective. Can this happen, because often technology is the only area where they can apply their intellect and creativity? Can this happen, because management doesn’t involved programmers in decisions related to business domain and understanding customer needs? Can this happen, because programmers are often disconnected from real business needs and goals of the projects?
Over-engineered solutions, which are done with fashionable technology, but without practical necessity are bad. However, if two prior points could become true – programmers work directly with customers, understand their needs and translate them into the software – programmers will not have time and desire for fancy technical solutions, but will strive for pragmatic, lean and focused on customer value solutions.

Theory P Corollaries

Mr. Bryce consider programmers as the herd of wild geeks who need strong management control. These management shepherds should control programmer’s lazy, pseudo-intellectual and capricious nature. And also software projects need the army of business analysts who can chew over for programmers business needs (as low IQ obviously doesn’t help programmers to get it directly). And certainly software projects badly need management consultants like Mr. Bryce who can tell how to properly treat programmers and create theories of their behavior.

There is another way. Treat and trust programmers as responsible human beings, let them work directly with customers and let them make most project decisions. It is not an easy shift in minds considering existing stereotypes from both management and programmers sides. But it could be done in The Ideal Software Company.

I have a dream about a company that always makes users happy. The company that quickly builds high quality, simple and usable software that fully meets user needs. The company that reacts rapidly on customer demands, anticipates new needs and immediately use these opportunities. The company where people are happy, motivated and could realize their potential, dreams and life goals. The company that is successful on market and in the hearts of users, investors and employees. The Ideal Software Company.

The Barriers

What are the main problems with building software? What are contributing factors to these problem? What prevents many existing software companies from being ideal?

1. Unclear, incomplete or misinterpreted user needs. The software programs often don’t solve needs on adequate level, have bad usability and contain many unused features.
2. Difficulty to estimate required effort. Missed deadlines, doomed schedules and budget overruns are the common deceases in software development.
3. Inability to adopt to changes. Often it is even impossible to estimate effort at the beginning as many things are changing during the course of the project: customer needs, performance requirements, business context. But for many software companies these changes still bring chaos to their projects.
4. Poor, rigid or low quality solutions. Implementation defines project success. Talent, experience or knowledge are necessary to build good solutions, and they are often missing in the project teams.

The main reasons for these problems

1. Top-down decisions – only few people make decisions with limited information, lack of feedback and independent information
2. Poor aggregation of information from different sources
3. Lack of diversity: homogeneous groups that are responsible for only specific aspect of the project (requirements, design, programming, etc.)
4. Lack of accountability for the end results, mismatch in company and personal employees goals, politics and suboptimal decisions.
5. Low motivation and morale – minimal control over decisions, poor relation of incentives and payout to contribution into the company success.
6. Centralization – authoritative control causes inflexibility, lack of independence and adaptability; missing local tacit knowledge in decision making.

James Surowiecki in his book The Wisdom of Crowds clearly shows that traditional top-down companies are inefficient in making the best decisions.

The Ideal Software Company

A. Company organization

1. Powerful teams – the company represents a conglomerate of powerful teams. Projects teams are small, diverse and empowered to make most project decisions.
2. Egoless management – management carry mostly support, coordination and representation functions. Leadership is moved to the teams level. Top management is a consulting body, which includes highly skilled professionals in finances, technology, sales, etc.
3. Shared and competing infrastructure – company infrastructure could include services as hosting, customer support, recruitment or distribution. These in-house services are shared and compete with outside providers.

B. User needs, product and feature ideas

1. Generate many alternatives – the company uses diverse sources (internal, user community, paid analysts and experts, etc.) for ideas generation
2. Predict most successful ideas – the company establishes prediction markets for deciding forecasting most successful ideas with large group of participants – customers, user community, domain experts, managers, designers, and developers. They make independent guesses based on their private knowledge.
3. Select the best ideas – any team could use this information to sign up for implementation of the product or feature ideas.

C. Team is a business unit

1. Full responsibility – a team is ultimately responsible for all decisions and end-results. The team selects product ideas, come up with solutions an implements them.
2. Diverse teams – teams are diverse, self-sufficient and include people with different background, areas of expertise: marketing, domain experts, UI designers, developers, system administrators, testers, customers and community representatives.
3. Organic composition – the team can choose its composition, hire (or outsource) an additional or remove poor performing member; selects in-house or outsourced services (hosting, customer support, etc.)

D. Team formation

1. Talented people – the company hires talented people with diverse background, skills and experience. The only requirement is high qualification, creativity or expertise.
2. Easy start – Anybody in the company can initiate and assemble the team for implementation. The team applies to the internal VC group (see below) for financing and approval of the project.
3. Easy scalability – the team can contribute to open-source projects (at least for non-core and non-competing features and modules) and easily scale beyond company borders.

E. Sharing and learning

1. Shared best practices – the company promotes the best practices, tools, designs, conventions and approaches from internal and outside sources. Teams are free to select and use any of them.
2. Cross-team training – the company organizes cross-team knowledge sharing and training.
3. Continuous improvement – the company encourages regular reflection and continuous improvement sessions, where the organization, processes, company and teams interactions are scrutinized and changed.

F. Financing, incentives, internal market

1. Internal Venture Capitalist group (VC) – the company has something like Venture Capitalist (VC) group, which includes management, investors representatives and leaders of top teams that approves and finances new projects and teams. Teams are starting with minimal budget that barely covers their first needs.
2. Team value market – The company has internal market for the value of each team. This value is linked to success criteria established by company: (revenue, number of users, cost saving, use inside company, buzz).
3. Pay based on performance – Team budget, payout and incentives are directly corresponding with their value. More successful teams will get more money, resources and influence in the company. VC group disassemble unsuccessful teams. While any employee has guaranteed minimal salary, the people who doesn’t contribute in any team will be fired. People who performs will be very well rewarded.

G. Mobilization

1. The company in time of emergency or especially large projects can force creation of special task teams (as National Guard). Company leaders should approve this temporary mobilization and limit impact on other projects

The Benefits

The Ideal Software Company should overcome most barriers.

• Teams are diverse, independent and make most decisions. Information flow is free and the company ensures that teams receive it most effectively.
• Individual income is directly linked to their team performance (but not individual contribution inside the team). Teams compete with market not between themselves.
• Teams can still uses benefits of centralized services, coordination of activities and shared knowledge. Company organizes prediction markets, internal VC group and provides financing of the projects.
• The system is adaptive, lean and focused on the market and user needs. Bureaucracy is almost absent, the company need only few coordination mechanisms, the approach is compatible with our human nature and self-interest. It is combination of adaptability of decentralized and power of coordinated system.

What if we have only one team? We receive a small, lean and powerful startup. As a startup becomes bigger, it is often turns into inefficient traditional corporation with hierarchical control. Many startups lose their flexibility and adaptability once they become bigger. The Ideal Software Company should avoid this fate with the growth.

Is it possible?

I see many potential problems with creating The Ideal Software Company – political (most managers are not egoless), economical (internal financing schema, appeal to investors), organizational (new ways to employ people). But I believe it is possible.

Agile development processes (Scrum, Extreme Programming, Crystal Clear) are changing the role of people and teams in software companies. I know few companies that are close to the ideas of The Ideal Software Company. Google in software world, 3M and Toyota in manufacturing. Google has free flow of ideas, powerful self-organized teams and attention to user needs. 3M has entrepreneurial cross-functional teams with focus on business goals. Toyota become #1 auto manufacturer because their Toyota Production System gives unprecedented power to the teams and continuous process of improvement and solving root problems.

Is The Ideal Software Company utopia? What do you think?