Home    Services    People    Process        More About Us

XP Distilled

Programmers have been churning out code now for over 50 years. In that time, mountains of code have been written -- some of it good, most of it bad. The reason for this poor average is simple: traditional approaches to software development set projects up for failure. The worst part is that well-meaning, hard-working, brilliant people see a significant number of their projects fail. In "XP distilled," we gave you some numbers on project success. Figure 1 shows some updated numbers as of the 2000 Group CHAOS Report. (Incidentally, my editor asked me if I would mind defining "CHAOS." I responded to him that I wouldn't mind in the least, except that the Standish Group refuses to divulge the acronym's meaning. And -- as I told my editor -- no, I'm not putting you on. You can read a blurb about it in Keeping CHAOS quiet, available in Resources.)

Figure 1. Software project success and failure, then and now

I first saw a graph similar to this one in a software magazine article. The authors titled the graph, "Projects Show Steady Improvement," and they noted that in 1994 only 16 percent of projects succeeded, whereas in 2000 28 percent succeeded. While 28 percent is certainly better than 16 percent, the results are still abysmal. As I said before, if you use standard software development approaches, even if you are developing Java applications, prepare yourself for disappointment. Despite some improvement since the first CHAOS Report in 1994, almost three-fourths of projects still fail. No wonder company leaders are reluctant to spend money on software projects.

Why are these numbers so bad? There are probably as many opinions on that question as there are people asking it, but I think there are a few primary reasons:

• People aren't convinced they have a problem.
  
  
• People know they have a problem, but are afraid to risk doing something different to try to solve it.
  
  
• People know they have a problem, are willing to try to solve it, but misunderstand the problem they are trying to solve.
  
  
• People know they have a problem, are willing to try to solve it, understand the problem, but are constrained to the status quo.

Let's cover each of these reasons briefly.

Not convinced there is a problem
People have an extraordinary capacity for self-delusion. Company and project leaders are no exception. It's quite possible for software development projects to be going down the corporate drain while everyone is convinced everything is fine -- or at least they act like it is. The numbers I mentioned earlier should make it obvious that software development in organizations isn't going well. There are exceptions, of course, but lots of organizations are oblivious to their own problems.

Does your organization have an adversarial relationship between IT and "business people"? Do business leaders in your organization say things like, "If technology is so slick, how come all I hear from IT is 'No'?" If so, you have a problem that needs solving. If you don't solve it, inertia may keep you going for a while, but failure looms.

Know the problem exists, but afraid to risk solving it
More commonly, intelligent and observant people in an organization recognize that their current software development methods aren't working. They simply are afraid to risk trying something different to make the situation better. This is understandable. Trying new things requires courage and often involves taking a risk. And in our culture of quick wins and instant gratification, failure can be harmful -- or even fatal -- to your career.

With the odds stacked against them, most people take the path of least resistance. This approach may preserve a career in the short term, but it only postpones and compounds the problems. If you wait too long, little problems become insurmountable. Ultimately there is no way to force people to be brave and take more risk, but ideas in this column and the associated forum might help you get people in your organization (including yourself) past the fear of failure.

Know a problem exists, willing to try to solve it, but misunderstand the problem to solve
Sometimes people know they have a problem and are willing to try to solve it, even if that means trying new things. But they try to solve the wrong problem. For instance, they try to solve a production problem, assuming (perhaps unconsciously) that developing software is like turning out products on an assembly line. It doesn't make sense to have folks on an assembly line diverge from the efficient order and method of assembly. Software is different because it is an emergent reality. In his presentation from XP2002, Agile Methodologies: Problems, Principles, and Practices (see Resources), Jim Highsmith describes the difference between optimizing and exploring in software. In software, we're often exploring new territory and doing things that haven't been done before. This makes software development an entirely different sort of problem that requires a different solution.

If you try to "mechanize" or control the software development process too much, you'll lose control. I hope the upcoming articles in this column will help to articulate the software development problems you are trying solve.

Know the problem exists, willing to try to solve it, but constrained by the status quo
Finally, people might know there's a problem, are willing to solve it, understand the problem they're trying to solve, but are unable to act within their organizations. This is a sad and difficult -- but not necessarily impossible -- problem to overcome. Unfortunately, it requires much more courage than most people have. Sometimes you have to step on some toes to cause change. I know that's easier said than done, but if you're part of an organization with leadership that refuses to address problems jeopardizing the organization's health, you have a choice to make: you can go for broke and try to create change, or you can leave before the organization gets crushed under its own weight. This article and future ones in this column should give you some good arguments to help overcome resistance to new things in general -- or to XP in particular -- within your organization.

One solution: Agile methods
In a way, the name "Extreme Programming" is unfortunate. Most people probably think of extreme sports -- or a Microsoft operating system -- when they hear about XP. The idea behind the name is that there are best practices for software development, such as writing unit tests and code reviews. Why not do them all the time? When you do, they morph into concepts like test-driven development and pair programming, which most people view as rather extreme. That's where the name came from. It has nothing to do with soda, bungee jumping, or Bill Gates.

Perhaps as a defense against misplaced and myopic criticism of the name, people who think approaches like XP are good have begun calling such methods agile. You'll hear talk of Crystal Methods, Adaptive Software Development, and (currently the most popular) XP as agile methods. All of these processes embrace the reality that people work with people to develop software. Successful software processes must maximize people's strengths and minimize their weaknesses. In my opinion, XP does the best job of addressing all the complementary forces that affect the people involved, and of helping the programmers on the team to do the right thing when they write code. It also lets them write code most of the time, which is what they like to do anyway.

Whether you call your approach agile or something else doesn't matter -- results do. The goal is to balance all the forces to develop software intelligently. In my opinion, this approach requires brutal honesty all the time. It requires radical commitment so you don't buckle under the constant pressure to quit and do things the old way. It also takes skill.

Contrary to what most people say, I do not think every programmer can flourish on an XP team. The XP approach requires too much constant honesty and humility. Programmers tend not to learn that in school, and most corporate environments don't reward it. Humans develop software. We all have weaknesses, and we share many of them. We need an approach that lets people be people, and makes it very difficult for them to behave badly. I believe XP does that better than any other approach I've ever seen.

The XP values
As we said in "XP distilled," XP prescribes a core set of values and practices that allow software developers to do what they do best: write code. XP eliminates the unnecessary artifacts of most heavyweight processes that distract from that goal by slowing down and draining the development staff (such as Gantt charts, status reports, multi-volume requirements documents, and so on). Kent Beck outlined the core values of XP in his book, Extreme Programming Explained: Embrace Change (see Resources). These values haven't really changed in the last year. I still summarize them this way:

• Communication: Problems with projects often can be traced back to somebody not talking to somebody else about something important at some point. XP makes it almost impossible not to communicate.
  
  
• Simplicity: XP proposes that you always do the simplest thing that could possibly work with regard to process and writing code. The way Kent puts it, "XP is making a bet. It is betting that it is better to do a simple thing today . . . than to do a more complicated thing today that may never be used anyway."
  
  
• Feedback: Concrete feedback early and often from the customer, from the team, and from real end users gives you more opportunity to "steer" your efforts. Feedback keeps you on the road and out of the ditch.
  
  
• Courage: Courage exists within the context of the other three values. They all support each other. It takes courage to trust that concrete feedback along the way is better than trying to know everything up front. It takes courage to talk to others on the team when that might expose some of your own ignorance. It takes courage to keep the system simple, deferring tomorrow's decisions until tomorrow. And without a simple system, constant communication to spread knowledge, and feedback to steer with, it's tough to be brave.

In a paper I co-authored for the XP2001 conference in Sardinia, Italy, I suggested that introspection be added to this list. I realized when I was presenting the paper there, though, that introspection was really more of a practice. The strong candidate for inclusion as another value is based on a paper by Joshua Kerievsky (see Resources). In it he talks about how continuous learning is a requirement for a healthy XP team. I agree. I don't know what kind of support there would be among XP practitioners for adding another value, but I think it makes good sense.

The need for revision
When I first read Extreme Programming Explained, I was so excited I could barely put it down. I was struck by the huge contrast between software development in most corporate environments I had seen and the refreshingly honest approach Kent was describing. But I had a sneaking suspicion that the book left a little bit too much to the imagination on the business side of projects, which would cause some problems with acceptance. I was struck by an overwhelming sense that XP is about much more than programming.

Fortunately, there has been significant talk in the XP community, Internet newsgroups, and other forums about the "12 practices" of XP. People are recognizing that the first release of XP, so to speak, seemed to emphasize the programming aspects of software development to the detriment of other things, and that the business side wasn't quite as complete as it needed to be. The foundational principles are good, but there are some more practices that need to be included to complete the picture. Implicit in the revised practices list (which contains 19 practices instead of 12) is the recognition that XP is, in fact, about more than programming. It is about creating the kind of organizational change necessary to produce great software. This approach requires impressive coding discipline and skill, certainly, but it also requires significant changes in the way everybody on the team thinks about creating software. This means everyone on the team, including users, business analysts, and strategists. The new practices make this absolutely clear and focuses on creating one team to produce great software in perpetuity. In fact, this emphasis on one team is the primary driver behind the changes taking place in the XP world. In a draft of a paper on the subject entitled One Team, Kent Beck said:

The biggest error in Extreme Programming Explained is the implicit assumption that you have a technical team serving a single customer.

Later in the same paper he said:

The problem is compounded in XP Explained (and subsequent writings and conversations) by referring to the programmers when using the phrase "the team," as in "the customer speaks to the team with one voice." This usage always bothered me, but I didn't know what to do about it.

The new practices and the focus on one team try to address that concern. Any approach that aims to bring about organizational change in software development must heal the rift between IT and business. One of the biggest reasons there is a rift in the first place is that people involved in creating software haven't been part of the same team. So what is this one team supposed to look like? In the same paper, Kent uses a three-legged stool analogy. The programmer team is the programmers, while the business (or customer) team is the analysts, users, testers, and so on. The third leg is the management team, which is made up of people who answer such questions as:

• How does the project get started?
• How is investment increased, reduced, or terminated?
• How are disagreements resolved that aren't handled by business and development?
• How are relative priorities set between this team and all the other projects that need to be completed?

So now we have our one team defined as three subteams: programmers, customers, and management.

While the management team makes some of the big business decisions, there isn't a single team in charge of the development effort. Everybody is on a single team, with different people playing different roles. Of course, this can be easier to say than do. It requires a good number of people to be willing to do things differently. There is a risk of failure (and possibly termination), but the payoff can be grand.

Great. We have one team. What does everybody do? Just like a year ago, all of the XP practices translate the four values into the things you should do every day as a member of a software development team, whether you're technical or not. There isn't much new here for people in either role. The XP programming practices have been recognized by the industry for years as best practices, and the more business-focused practices are recognized as effective (see the Standish Group CHAOS Report in Resources for evidence). It is still true that the word "Extreme" in Extreme Programming comes from two things:

• XP takes proven industry best practices and turns the knobs up to 10.
• XP combines those practices in a way that produces something greater than the sum of the parts.

If everybody is on the same team, the second point can become amazing. But as soon as you cut off one leg of the stool, prepare to meet the floor. XP has never guaranteed -- and never will guarantee -- that people will do the right thing all the time, but it allows them the opportunity to try. This goes for all members of the team, technical or not. People on one team using all of the practices working together can produce significant increases in speed and efficiency. These practices can create an environment that allows an organization to produce great software. Should there be any other kind?

Revised practices
The 19 practices of XP (see Table 1) define the habits various members of the team should have. This month, we'll take a closer look at the joint practices, since they bind all of the members of our one team together. Next month, we'll look at the programmer practices, which formed the bulk of the original 12 XP practices. The month after that, we'll cover the customer and management practices. This will give you a better understanding of what it means to develop software with XP.

Table 1. The 19 practices of XP

Joint practices	Iterations Common vocabulary Open workspace Retrospectives
Programmer practices	Test-driven development Pair programming Refactoring Collective ownership Continuous integration YAGNI
Management practices	Accepted responsibility Air cover Quarterly review Mirror Sustainable pace
Customer practices	Story telling Release planning Acceptance testing Frequent releases

Before we discuss the joint practices, let me be clear about what the practices are and are not. The practices are not XP. XP is more than that. In fact, the XP values aren't XP either. Ken Auer, Erik Meade, and Gareth Reeves wrote a paper called "The Rules of XP" (see Resources), which described this well. In a nutshell, they said:

XP's values make it agile. XP's practices do not define XP . . . XP is defined by its rules.

Using Alistair Cockburn's game analogy, they go on to describe the "Rules of Engagement," which dictate the environment in which software development can take place effectively. They then discuss the "Rules of Play," which define the minute-by-minute activities and rules within the framework of the Rules of Engagement. They suggest that:

• The Rules of Play are what make XP unique.
• Following the Rules of Play is Extreme Programming.
• Following the Rules of Play and the Rules of Engagement is Extreme Software Development.

The XP practices are important because they reinforce behavior that increases the chances of a team producing great software for as long as people on the team want. But XP is more than that. It extends beyond programming to encompass the entire software development process. The revised XP practices take this into account by requiring different behavior from everyone involved.

It is becoming increasingly clear that there are four categories of XP practices. There is a set for each team within the one team, and there is a set that brings those three sub-teams together. Most of the original 12 practices are in the programmer team set, but some will probably be moving. Some names have changed, mostly because the original names either weren't as good as they could have been or didn't really make as much sense as they should. Some practices weren't in the original list of 12. In the next section (and in future articles), after each name, I'll parenthetically note whether the practice is new, is unchanged, or maps to an original name. Note that these names are in flux, but the principles probably aren't.

Before describing the practices, it's important to address what I would call the "piecemeal argument": does a team have to use all of the practices, or can it just pick the ones it likes? XP as a discipline assumes a team is using all of the practices all of the time, but a team can pick a few practices and use them successfully (such as refactoring or pair programming). You are free to do that on your team. I would choose not to do that on my teams, because I believe the practices are mutually reinforcing and work together so well that I give up something amazing if I leave one or more out. But that decision is up to you. If you want to try XP, I recommend you use all of the practices all of the time for a while, to see which ones you would be willing to give up. I'll bet the list will be small.

Okay, on to the practices. I'll talk about the joint practices this month to set the stage for all the others. First we need to make sure we have a single team using all of the practices together. This is a prerequisite.

Joint practices: Creating one team
This group of practices is for everyone. Everybody on the team (programmers, customers, and management) needs to be doing these all the time. They bring the sub-teams together to create the one team we're after.

Common vocabulary (Maps to metaphor)
The original metaphor practice was a dud. The idea wasn't bad: to have a controlling image that describes the system in terms everyone can understand and gives it a unifying theme that allows the team to know where new things fit. The problem was that the practice really distracted from the main goal: a shared understanding of the system. Everybody on the team needs it. Sometimes a great metaphor isn't available, or would take lots of time to figure out. Don't delay simply because you don't have a nice word picture of the system you're trying to build. Focus on having a shared vocabulary (what Ward Cunningham calls a "System of Names") that everyone can use. If a metaphor helps, use one. If you can't find a good metaphor, just agree on the shared vocabulary (which shows up in names for classes, methods, and so on).

In Extreme Programming Explored (see Resources), Bill Wake talks about metaphor as well. He says:

Determine the metaphor during the exploration phase, when stories are written and spiked. Revise it as you go. Let the metaphor guide your solution. Use its names for the uppermost classes. Understand how those classes interact.

He suggests using the "naive metaphor" (that is, let objects be themselves) when you can't think of a better one. Good advice.

Open workspace (New)
Informal communication is more effective than formal most of the time. Cubicle walls make informal communication more difficult. The best solution is an open workspace where people can overhear things, contribute when it makes sense, and have impromptu discussions about anything that matters. This kind of group interaction can create amazing emergent results. I've seen it in action. Tear down the walls. Your team will be better for it. If your organization can't stand to have some furniture moved, you have a bigger problem.

A typical reaction to the open workspace practice is skepticism that a bunch of people talking in the same area would distract each other. Speaking from experience, that does happen. An open workspace works well when the people on the team respect each other enough to work in that workspace, rather than goofing off. If someone wants to goof off, they go to the goof-off area, which should be a different place. This doesn't mean the workspace can't be fun. It should be fun, but it also should be a place where work gets done. If the members of a team care about each other and care about the work they're doing, they can make the open workspace work. You should assume it won't be a problem and only handle it if it becomes one.

I've heard some people suggest that the open workspace practice means XP will not work for large teams. They suggest that a large number of people in a single mega-cube (or some other open workspace) would be chaotic. Maybe they're right. I think that's the wrong emphasis, frankly. They seem to be assuming two things: that you need a large team and that there isn't a collaborative environment that will support one.

Perhaps you need a large team. But did you really think about it, or did you just grow? Perhaps a small group of good developers working in an environment conducive to doing great things -- with the support they need -- can do more, better and faster than a larger team. Maybe not, but it's worth considering.

Suppose you do need a large team. How could you create a collaborative environment for it without creating utter chaos? What if you had multiple adjacent open workspaces with semi-permeable barriers (an open doorway, a half-wall, an aisle between mega-cubes, and so on)? I believe most people who say you can't create a collaborative environment to support a large team aren't too fond of the idea generally -- small team or large. It's a mistake to assume that an open workspace precludes a large team.

Iterations (New)
People who use XP often talk about the "rhythm" of a project. This rhythm exists at different levels, but the iteration is the real heartbeat. Every one to three weeks the team delivers a working system with new features the customer requested. That's a radical idea. People are not used to creating software this way. Most approaches focus on doing lots of stuff before you write code. XP requires that you write code, let people use it, and let their feedback steer the project (we'll discuss this concept more in a future article).

Each iteration begins with some planning. I (and lots of other people) call this iteration planning, and we do it in an iteration planning meeting. The planning that goes on in this meeting looks a lot like release planning (again, we'll discuss this concept in more detail in a future article), but at a more detailed level. We're concerned only with the next iteration. We ask the customers, "If the project were to end after the next iteration, what features would you want to have?" Then we estimate the work to build those features. We fit what we can in the iteration and defer the rest. Customers drive the priority; the programmers specify the cost. That means some things have to fall off the plate for the current iteration. You can't compromise on this. You can't make more time.

A word about timeboxing is important here. Timeboxing is the practice of setting a time limit for some bit of work, working until the time's up, and then assessing where you are. It is a technique for preventing the team from working too long on tasks, to the exclusion of other things that need to be done. The iterations practice is about more than timeboxing. The iterations practice is really about requiring people to make tough decisions that they are most qualified to make. Customers make decisions about which features are most important to build next. Programmers make the technical decisions about how to build features and how much effort that will take. Management makes decisions about strategic direction for the project and how it needs to fit into the rest of the organizational picture. Everybody on the project team is involved, but they stick to what they know. They use their brains, of course, but they also trust others to use theirs.

There has been some discussion about what to call this practice. Should we call it short iterations to emphasize the shortness of the iterations, or iteration planning to emphasize the community aspect of bringing everybody together to plan? I favor short iterations because it emphasizes the need to get real, working software in users' hands to get concrete feedback.

Retrospectives (New)
It's been a rare project in my career where everybody on the team took a little time to look back and identify openly what really stunk. I've had to do that myself. When I did (if I had time), I usually discovered that the problems with a project usually could have been solved before they became problems if the team had spent just a little time addressing them. That's what retrospectives (a term coined by Norm Kerth) are about.

Working without reflecting on what you're doing locks you into repeating your failures. It would be better for each person on the team to reflect on his own actions and on what he's learned, then share that with the rest of the team. As I outlined with Chris Collins in our XP2001 paper Adaptation: XP Style (see Resources), the individual behavior I'm talking about is introspection. You make your introspection public in a retrospective.

Think about lessons you've learned on the project. Think about what you did well and what you screwed up. Think about how the team functioned. Think about what the team produced collectively. Could things have been better? Why? More importantly, how? What things did you do well that you want to make sure you keep doing? How? This kind of collective review should happen in an organized way at least after every release, preferably after every iteration. It doesn't have to take long. Reserve the first hour or two of your iteration planning session for a retrospective on the previous iteration. Perhaps reserve a day (or more) for a post-release retrospective. Don't pull any punches. You're trying to make your team better here, trying to increase the value you can add to the organization, trying to make your own life better by putting the fun back into work. This is vital.

Most of my colleagues and I have found it helps for each pair on the team to have a "mini-retrospective" at the end of each day. Just a few minutes to reflect. If we learned something worth sharing, we mention it in the team meeting the next day. We've learned a lot this way.

Retrospectives certainly require some vulnerability to be effective. The compensation is a more integrated team. Everybody needs to take part in the retrospective, both the technical and non-technical people. If this would require too many people to meet in one place (and it might), subteams (such as management) could have their own retrospectives and send a delegate to a smaller meeting. The point is to learn so we reduce our chances of making the same mistakes over and over again. This is one team and everybody needs to be fully invested in it.

Organizational change
Ultimately, XP is about organizational change -- it simply starts with software. Putting everybody on the same team, but giving them different jobs, is the best way I can imagine to create change that works. This month we looked at the joint practices of XP that help create the one team we need in order to create that kind of real, lasting organizational change.

As I'll say in closing every time for the next few months, it's important to reiterate that the revised practices I've described here are not necessarily what the "new" XP will look like. I imagine we'll all have to wait for the second edition of Kent Beck's Extreme Programming Explained: Embrace Change to find out what the next release looks like.

Before we get into the programmer practices, let me clarify something from last month. The revised/renamed/new practices I'm talking about in this column represent my musings on two unpublished articles written by Kent Beck. There haven't been any official changes to XP, as far as I know, and these practices and ideas haven't appeared in any formal way on the leadership group or similar location. Maybe there will be no official changes. There certainly isn't any kind of formal revision effort underway. I think the most I can say at the moment is that what I've written might be XP at some point in the future. It'll probably look different. I hope my ideas contribute to the discussion. Read at your own risk.

Test-first development (Maps to testing)
Whenever programmers change their code, they need to know if what they just did is an improvement or if they just broke something. More importantly, they need to maintain the discipline necessary to create the smallest amount of code necessary to get the job done, not some bloated behemoth that has a bunch of hooks somebody might need later. This is what test-first development is all about.

There are two kinds of testing in XP: unit testing and acceptance testing. These are the typical names, but I don't like them. They're too much jargon for me. I prefer the names suggested in "What is Extreme Programming?" (see Resources) by Ron Jeffries: customer tests and programmer tests.

These names get to the heart of the reason behind the two kinds of tests. Programmers write the programmer tests as they write code. Customers write customer tests after defining stories. Programmer tests tell developers whether the system works (as defined by the customer) at any point in time. Customer tests tell the team whether the system does what users want it to do. I'll talk about programmer tests here, and cover customer tests next month.

Assuming the team is using an object-oriented language like the Java language, developers write programmer tests for every method that could possibly break (just the public interface most of the time), before they write the code for that method. Then they write just enough code to get the test to pass. People sometimes find this a little weird, but the point is simple. Writing tests first gives you:

• The most complete set of tests possible, which increases your confidence in the code
• The simplest code that could possibly work, which makes it easier to refactor later
• A clear vision of the intent of the code, which makes it easier to understand and refactor later

A developer cannot check code into the source code repository until all the programmer tests pass. Programmer tests give developers confidence that their code works. They leave a trail for other developers to understand the original developer's intent (I've rarely seen better code documentation). Programmer tests also give developers courage to refactor the code, because a test failure tells the developer immediately if something's broken. Programmer tests should be automated and give a clear pass or fail result. xUnit frameworks (see Resources) do all this and more, so most XP teams I know of use them. There is an xUnit for almost every language imaginable. Just replace "x" with the language or tool of your choice (for instance, VBUnit for Microsoft Visual Basic, CppUnit for C++, and so on).

As a programmer, I am frequently amazed by the difference between coding with programmer tests and coding without them. I find myself taking extremely small steps when I write code. In fact, I don't have to debug very much when I'm taking small enough steps, because the source of a problem becomes obvious: it's got to be that line of code I just wrote. The phenomenal thing to me is that writing a test first quite often drives me to create the simplest code possible.

I'm sure you have written code you thought you would need later, but did you need it? Perhaps, but if you didn't, did you remove it? Probably not. So it just sits around not serving any useful purpose. What if your test told you exactly how much code to write? What if you only wrote enough code to get the test to pass, and no more? You would probably have less code, and it would all be used. That's the kind of code I want to write. When you write your test first, the test drives the code. As a result, your code will look remarkably different when tests drive it. And in my experience, my code is much simpler when tests drive it, and I'm always in the market for greater simplicity. Kent Beck is writing a book called Test-Driven Development (see Resources), which details this practice. I recommend it. In fact, I prefer the name test-driven development for this practice.

Pair programming (Unchanged)
In XP, pairs of developers write all production code. Most developers have never experienced actually writing code with another person, and it can feel a little strange at first. Pair programming typically means two developers share a single computer. One person keys in the code (the "driver") and the other helps him find his way (the "navigator" or "partner"). If the driver gets stuck or frustrated, or if the navigator has a good idea he can't describe too well without typing, the current driver can give up control and be the navigator for a while. Each person should switch roles frequently. Once you get used to these roles, switching back and forth happens rather freely.

This approach may sound inefficient. I've always liked Martin Fowler's response: "When people say that pair programming reduces productivity, I answer, 'That would be true if the most time-consuming part of programming was typing.'" In fact, pair programming, or just "pairing" for short, provides many benefits, including:

• All design decisions involve at least two brains.
• At least two people are familiar with every part of the system.
• There is less chance of both people neglecting tests or other tasks.
• Changing pairs spreads knowledge throughout the team.
• Code is always being reviewed by at least one person.

Empirical research indicates that code reviews increase code quality, but I hate doing them. I also believe they tend to be very difficult to do correctly, and they aren't nearly as effective as pair programming. What if every line of production code were reviewed by somebody who was intimately familiar with what the code was supposed to be doing? Pair programming gives you precisely that. Nothing keeps you honest like a witness.

Research discussed in The Costs and Benefits of Pair Programming by Alistair Cockburn and Laurie Williams (see Resources) is also showing that programming in pairs is actually more efficient than programming alone. This is a bit counterintuitive. Most managers (and I've been one) will see two developers doing the work of one and stop there. That's not thinking beyond the end of your nose. It's too simplistic. It's also false.

As for risk, think for a minute about why projects fail. One of the big reasons is extreme dependence on individual heroes. If your project's hero is killed in a freak farming accident, your project might be toast. The essence of pairing is to spread knowledge around. Pairs should switch around every so often. If pairs get too sticky, they get stuck.

Despite the good numbers and the strong arguments in favor of pairing, however, most developers hate this idea. Perhaps it's an issue of pride. Deep down, I believe most developers want to be the hero. Pairing makes that next to impossible. As I wrote in Extreme Programming Applied: Playing to Win (see Resources):

Anybody can sit next to someone else and throw in two cents every so often. Many people can be completely engaged and try to make the result better. But the ones who really understand pairing know that it's about loving another person.

As I said in the book, I'm talking about the kind of love that shows itself through actions, not the romantic kind. When you love another person, you try to see the best in him and help him grow. You'll be patient, kind, not jealous, generous, humble, polite. This kind of intense interpersonal relationship, even if it's just for a few hours, is not something most people are interested in, and developers are people. This is a radical idea and it isn't for everyone. But it also can produce some of the most rewarding experiences of your professional career.

Refactoring (Unchanged)
Refactoring is the technique of improving code without changing functionality. An XP team refactors mercilessly. Two key opportunities exist for developers to refactor: before and after implementing a feature. Developers try to determine if changing existing code would make implementing the new feature easier. They look at the code they just wrote to see if there is any way to simplify it. For example, if they see an opportunity for abstraction, they refactor to remove duplicate code from concrete implementations. An important thing to note here is that you should either design and write new code, or refactor existing code. Don't try to do both at once.

While XP says you should write the simplest code that could possibly work, it also says you'll learn along the way. Refactoring lets you incorporate that learning into your code without breaking the tests. It keeps your code clean. That means it will survive longer, introduce fewer problems for future developers, and guide them in the right direction.

The point of refactoring is to improve the design of existing code. The team should have automated suites of programmer tests and customer tests. The former should pass all the time; the latter should pass to the degree that customers require. Run the tests. Refactor the code. Rerun the tests. Did any programmer tests break? Fix them. Did any of the "required" customer tests break? Fix them. If you can't fix it, back out the refactoring you just tried. Without tests, changing your code would be a guessing game. With refactoring, if the code breaks, your tests will tell you.

Collective ownership (Maps to collective code ownership)
Any person on the team should have the authority to make changes to any of the code to improve it. Everybody owns all the code, meaning everybody is responsible for it. This technique allows people to make necessary changes to a piece of code without going through the bottleneck of an individual code owner. The fact that everybody is responsible counteracts the chaos that ensues from no code ownership.

Saying that everybody owns the code isn't the same as saying nobody owns it. When nobody owns the code, people can wreak havoc anywhere they want and bear no responsibility. XP says, "You break it, you fix it." The team should have programmer tests that must run before and after each integration. If you break something, it's your responsibility to fix it, no matter where it is in the code. This requires extreme discipline.

I've noticed that some team members simply can't handle this practice. They can't stand the idea of somebody messing with "their" code. If you want to be part of an XP team, you must share ownership of all the code. If you don't, the team will eventually hit a wall. Some parts of the code will be off limits, which makes the system difficult to change -- a scenario we're trying to avoid. If somebody behaves this way, the team needs to point this out and encourage him to change. If he refuses, strongly recommend he change or leave. If he still refuses, tell him to move on. Don't compromise on this.

Continuous integration (Unchanged)
Integrate new changes into the system multiple times each day, and rebuild the entire thing automatically. Don't release changes until all the tests run. If a test fails, you have two choices: fix it and integrate, or don't integrate. A single failing test that you can't fix means you should not integrate.

Continuous integration does not mean you integrate every second, but it does mean you should integrate early and often. Daily is not enough. In an eight-hour day, I start to get that icky feeling if I haven't integrated at least once every couple of hours, if not more. This sounds scary to many people. Once again, the tests should drive that fear away. The tests tell me whether an integration "works" and that it's safe to release to the rest of the team.

YAGNI, or, "You aren't going to need it" (Maps to simple design)
In "XP distilled," Chris Collins and I wrote that "XP's detractors claim that the process neglects design." They still do, and they're still wrong, but I think XP fans sometimes gloss over the objection too quickly.

What this objection is really saying is that people aren't comfortable with the idea of emergent design: letting the design of the system emerge, rather than trying to nail it all down at the start. Establish a direction, note some milestones and landmarks, then start the trip. You can adjust along the way as you learn. People who don't like this approach mistakenly suggest it is undisciplined. In reality, it is the only realistic approach to development in today's economy.

Typical heavyweight methods say you should do all but the most trivial design tasks up front. This is like taking a static picture of the horizon, staying still, and trying to draw a perfect map of how to get there. This is a fine approach if requirements are constant. If you know at the beginning what the system needs to do and how it needs to do it, you can do most (if not all) your design up front. In reality, however, most developers are exploring problems that haven't been adequately solved before or are implementing solutions that haven't been tried before. These days, most systems are being designed for businesses competing in markets that change constantly, not once every ten years.

In environments like this, requirements are effectively changing all the time and stability of requirements is a pipe dream. This means big, up-front design is inappropriate. You simply cannot know at the beginning where you will end up in the end, or even where you want to end up. The best you can do is establish a general direction and make small, frequent adjustments along the way in order to hit a moving target. XP requires simplicity at every step so that you can change direction as often as necessary. We always try to use the simplest design that could possibly work at any point, changing it as we go to reflect emerging reality.

According to Kent Beck, the simplest design that could possibly work is the design that:

• Runs all the tests
• Contains no duplicate code
• Clearly states the programmers' intent for all code
• Contains the fewest possible classes and methods

Requiring a simple design doesn't imply that all designs will be small or trivial. They just have to be as simple as possible and still work. Don't include "extra" features that aren't being used. We call such things YAGNI, which stands for "you aren't going to need it." In other words, don't design for things you might need. In the next iteration, you may find out you didn't need it after all. Instead, write a test, then write just enough code to get that test to pass. That's usually all the design you need to do. Applying the YAGNI principle doesn't mean you can't ever think ahead, but it does mean you shouldn't look too far.

Practice synergy
All of the XP practices work together and are mutually reinforcing. This is especially true for the programmer practices. Some programmer practices, however, can stand alone (for instance, test-driven development and pair programming), while others (refactoring, collective ownership, continuous integration, and YAGNI) are more "dependent" practices, requiring other practices to be in place before they can work.

As I said before, I would be nervous about refactoring without tests. Likewise, without tests to tell me if I broke something, I would find collective ownership insane and continuous integration impossible. As for YAGNI, I don't think it's possible to keep your design simple without having tests drive you to that simplicity. The urge to build in hooks for the future is just too strong to resist. Interestingly, notice that most of these dependent practices are dependent on having tests in place. (Perhaps that practice is particularly important.)

While some of the practices can stand alone, most cannot. If you have tests, perhaps you could adopt more of the other practices in a piecemeal fashion, but why would you want to? Using the practices together and letting them reinforce each other can produce startlingly successful teams and even better software. If you don't think you'll like a particular practice, resist the "Green Eggs and Ham" defense and try it for a while. You might be surprised. If you have tried a particular practice and still don't like it, you can try to do the rest without that one, but I believe your speed and your results will suffer.

My recommendations from "XP distilled" remain the same: The whole is greater than the sum of the parts. You can implement single practices or a small subset, and get great benefits over not using any. But you only get the maximum benefit if you implement all of them, because their power comes from their interaction. Do XP by the book at first as a benchmark. Once you understand how the practices interact, you will have the knowledge you need to adapt them to your context. Remember that "doing XP" is not the goal; it is a means to an end. The goal is to develop superior software quickly. If your process mutates in a manner that disqualifies you from saying you are doing XP, yet your results are blowing the doors off your competitors, you have succeeded.

Customer and management practices

Level: Introductory

Roy W. Miller (rmiller@rolemodelsoft.com)
Software Developer, RoleModel Software, Inc.
October 2002

Roy Miller completes his review of XP practices by exploring the customer practices and management practices. The customer practices address the issues of determining which features should be in each release. The management practices help management give business direction to the entire team and keep them focused on the problem at hand. With the programmer practices and joint practices discussed in previous articles, you'll now have the complete picture of XP practices.

Over the last couple months, we've talked about joint practices and programmer practices. Joint practices pull all of the members of our "one team" together; everybody needs to do these all the time. The programmer practices detail the practices that programmers need to follow; while others may care about those practices, programmers need to live them daily.

This month we'll explore the customer practices and management practices. As with the programmer practices and joint practices, some of these practices weren't in the original XP list of 12. In the following sections, after each name, I'll parenthetically note whether the practice is new, unchanged, or maps to an original name. Note that these names are in flux, but the principles probably aren't.

Customer practices
The customer team doesn't care about programming per se, but they do care about the inputs and outputs. Customers determine which features should be in each release, what each of those features means, and whether the programmers can say they are "done" coding a feature.

Story telling (New)
Typical software development projects I've been part of specified requirements in rather formal documents that everyone referred to as "the requirements doc." XP is different for one primary reason: you rarely know all the requirements at the beginning of a project (and they change frequently anyway as the team learns).

Static documents that feel like contracts don't fit into the XP scene very well. XP teams learn as they go. This doesn't mean you shouldn't think before you start to code. It does mean you should be realistic about how far into the future you can think -- not very far at all. So, should we give up planning? No, but we need to plan differently. The release planning practice covers this, but with XP there's a prerequisite that involves a new way to communicate: story telling.

Think back to some of the requirements gathering exercises you've participated in. You might have conducted interviews, held rather lengthy meetings, and so on. You probably asked some clarifying questions and probed for more detail. At the end, you created a document and sent it around to everybody that mattered (and some people who didn't, but had to get the document anyway for political reasons). They were supposed to review it and sign off that you captured all the requirements. Did that go well? The only time it ever did for me was when the system was trivial, if then. Most of the people who had to sign off on the requirements felt nervous about doing it. They often said things like, "I'll sign off for now so we can proceed, but I'm not sure you captured everything." That made me nervous as I imagined the eleventh-hour revelations about critical requirements we missed. It simply wasn't a recipe for success.

The XP approach is radically different. It proposes that a team sit together in a room and talk about what the proposed system needs to do and how it needs to do it. Programmers or customers write these things down on note cards. Each note card says something like:

As a <some user role>, I need the system to <functional requirement> so that I can <business reason for the requirement>.

The stack of note cards becomes the set of requirements for the system, as they are known right then. The cards don't have exhaustive detail on them. Rather, each card is a promise for a future conversation to flesh out the details. In the meantime, the card simply needs enough meat on it to give the programmers an idea of how much work it will take to build it, and to give the customers enough detail to write an acceptance test (see Acceptance testing).

This form of requirements gathering requires a different skill from the business people who are the source of the requirements. In my experience, people in requirements meetings tend to throw any old thing out there just to go on record as having said it. That way, if something gets missed, it isn't their fault. This is bad for the team and is a significant cause of project failure. What we really need is requirements contributors who are good at telling stories. They tell them at the right level of detail, and they take responsibility for being the one to flesh them out when a pair of developers actually tries to code them.

Some people criticize this XP practice as too informal. I assume they mean that the requirements documents most projects have contain more formal language, each requirement has a reference number, and the requirements gathering process is more focused on arranged meetings with various stakeholders. In my experience, none of those things leads to better -- or more complete -- requirements. In fact, my experience tells me they lead to junk that's outdated as soon as it hits the page.

I have worked on successful projects with such formal requirements documents and requirements gathering methods, but my gut tells me the formal requirements document and process had nothing to do with our success. My teams still had to follow up on each requirement when it came time to code something. We still had to ask the source of the requirements to tell us the story XP says they should tell us to begin with. Why not shorten the process and start with the story? That's what this practice is about. Story telling is a skill, and some people are better at it than others, but everybody who contributes requirements to an XP project has to do it. Fortunately, everyone doing it can improve with practice.

Release planning (Maps to the planning game)
I never much cared for the "game" part of the planning game. It didn't sound professional enough, not that there is anything wrong with having fun at work. Release planning is about making sense out of the stories programmers and customers tell. Release planning happens once per release.

Some people like to criticize XP as glorified hacking -- just a bunch of cowboys cobbling together a system without any discipline. Wrong. XP is one of the few methodologies out there that recognizes that you can't know everything when you start. Both the customers and the developers will learn things as the project progresses. Only methodologies that encourage and embrace such adaptability will be effective. Status quo methodologies ignore change. XP listens. The way it listens is through release planning.

The main idea behind this practice is to make a rough plan quickly and refine it as things become clearer. The artifacts of release planning are:

• A stack of index cards, each containing a customer story, which will drive the project's iterations
  
  
• A rough plan for the next release or two, as described in Planning Extreme Programming by Kent Beck and Martin Fowler (see Resources)

The critical factor that lets this style of planning work is to let the customer make business decisions and allow the programmer team to make technical ones. Without that, the whole process falls apart.

Programmers determine:

• Estimates of how long it will take to develop a story
• Cost implications of using various technology options
• Team organization
• The "risk" of each story
• Order of story development within an iteration (doing risky items first can mitigate risk)

Customers determine:

• Scope (the stories for a release, and the stories for each iteration)
• Release dates
• Priority within each release (which features get developed first, based on business value)

Planning happens often, which provides frequent opportunity for either customers or programmers to adjust the plan as they learn new things.

Acceptance testing (New)
As I mentioned in the test-first development practice section of last month's installment, acceptance testing is the customer half of the XP testing picture. In fact, I prefer to call these tests "customer tests" because they focus on who should be creating it. Customers make sure each development story has customer tests to validate it. Customers can write the tests themselves, recruit other members of their organization (for example, QA people or business analysts) to write them, or combine the two approaches. The tests tell them if the system has the required features and whether the features work correctly.

Ideally, customers will have the customer tests for the stories in an iteration written before that iteration is over. Customer tests should be automated and run frequently to ensure developers are not breaking any existing features as they implement new ones. Typically, customers will need some help from the programmer team to write these tests. On one project, we developed a reusable automated customer test framework in Ruby, which in my opinion is the best possible language for the job (unless I can figure out a way to do it in Smalltalk). This framework lets the customer specify "actions" that the system should perform. The syntax is quite English-like. The interesting part is that the test scripts customers write are executable Ruby code. The Ruby framework reads each script, executes each action, and spits out "PASSED" or "FAILED" for each. The framework handles Web applications and desktop applications written with the IBM Standard Widget Toolkit (SWT). Customers have seen the value quickly. My company, RoleModel Software, is planning on making the framework open source.

Not all of the customer tests have to pass all the time. The customer decides which ones are critical and which ones aren't -- that's a business decision. The point is that these tests help customers gauge how "done" the project is. They also allow customers to make informed decisions about whether something is ready for release.

Frequent releases (Maps to small releases)
I have a tendency not to want to let people see stuff I'm working on until it's polished. That's silly. Getting feedback early and often keeps me from wasting effort on something nobody really wants. Releasing frequently gets working software in the hands of real users so they can give you the feedback you need. Releases should be as frequent as possible while still delivering enough business value to make them worthwhile.

Release the system as soon as it makes sense to do so. This provides value to the customer as early as possible (remember that money today is worth more than money tomorrow). Frequent releases also provide concrete feedback to developers on what meets customer needs and what doesn't. The team can then include these lessons in its planning for the next release.

Why is this a customer practice? Because the customer team needs to drive what the programmer team releases. The customer team is the one that knows whether certain system functions are useful for users at any point in time. Also, making the customer team responsible for frequent releases keeps them focused on the reason you developed the software in the first place: to deliver a useful product that people will in fact use. The best way to do this is to let your users tell you what they want. Most often, they can't do that very well if they don't have something to see and touch. Give that to them and they'll give you the feedback you need.

Defunct customer practice -- On-site customer
You may have noticed that one of the original XP practices related to customer behavior (although it wasn't categorized that way explicitly) is no longer in the list: on-site customer. Customers are now part of the "one team." It is assumed they will be on site and close by.

In my own experience, I have found it vital to be close to the customer. Ideally, I'd like the customer to sit in my cubical area, close enough that he can overhear what the team is saying. That would allow us to ask a question immediately whenever it comes up. Unfortunately, this kind of intimacy can't always happen. The next best thing is to have the customer close by, which means the programmer team has to be located with the customer. It is almost impossible to use XP effectively when you aren't close enough to your customer to ask questions immediately, or at least very soon. We've tried to have a remote customer who is available by phone "anytime," but it just doesn't work as well.

Management practices
When most people hear the word "manager," they probably think of somebody who tells people what to do. For a software project, management creates work plans, recruits developers for the "team," and keeps everybody on task, on schedule, and on budget. On an XP team, management is also part of the team, but in a different way than you (and they) might think.

Accepted responsibility (New)
Good managers rarely have to tell anybody what to do. Instead, a good manager points out problems that the team may not have noticed (because they're in the middle of it), then lets the team figure out ways to solve them. For many managers, this is a radically different way of thinking. As Kent Beck said in his paper on the revised practices:

This [practice] is a big shift for management, as it implies giving up control yet still being responsible for the team working effectively.

Having been a project manager, I know this approach sounds a little scary. But driving people like slaves -- or treating them like children -- sounds worse to me and is an approach that is likely to fail. It is very difficult to force somebody to do things your way. Instead, I would rather raise issues, make people confront them, and then let them figure out how to solve the problems. That way, you let developers and customers accept responsibility without abusing them.

Air cover (New)
When the Allied forces in World War II were pushing eastward through Europe after D-Day, one of the biggest factors that contributed to their success was splendid air cover to support the ground troops. The bombers and fighters pummeled the enemy and softened up the defenses so that the Allied ground troops had less work to do. While people who aren't on the project team aren't really the enemy, they can easily distract a team from its ultimate goal of planning, coding, releasing, and getting feedback. Management has to handle these interruptions.

There are also a lot of administrative and organizational things that get in the way of a team making software, such as signoffs for access to certain environments and getting broken workstations fixed. Management needs to remove those obstacles.

Finally, the team fits into an organization somewhere, and those other groups need to know what the team is doing and why. Management needs to be responsible for this communication.

Quarterly review (New)
Management should take part in the retrospectives I discussed in Part 1 of this series. The quarterly review is different. Its purpose is to get as much of management together as possible to make sure everyone (the team and managers) has all the information they require. This includes any managers who spend a lot of time with the team. It also might be wise to invite other people perhaps farther up the chain who may not spend as much time with the team, such as the manager's boss or the person paying for the project. Having a regularly scheduled meeting makes this kind of communication a regular part of life.

Mirror (New)
Perhaps management of an XP team needs to discover the lost art of leadership. Management needs to give the team an unvarnished picture of where they are at any point, through metrics, words, or both. If the developers or the customers get stuck, management can suggest ways to break the log jam by pointing out problems, making suggestions, giving advice, or providing encouragement. Only rarely should this practice include telling people what to do.

Sustainable pace (Maps to 40-hour week)
Without a pacesetter, the team is likely to burn itself out. Management needs to set the pace for the team. If people are working too hard, management needs to get them to slow down, go home, and get some rest. Tired people produce bad software. The number of hours isn't important; the level of fatigue is.

Management also needs to make sure the team has the time it needs to do what it needs to do. For example, many people consider refactoring a waste of time. In fact, keeping your code clean will allow you to keep your speed up; letting it get dirty will slow you down sooner or later. Management needs to make sure the team gets the time it needs for refactoring and other necessary tasks.

Copyright © 2001 by Chris Collins and Roy Miller, all rights reserved