I used to be plagued by nagging concerns about certain omissions and shortcomings of extreme programming (XP). Over recent months, I have come to the conclusion that my thinking on the topic was flawed; I used to criticize XP for not addressing a variety of issues such as the integration of testers, domain modeling, story generation, etc.

It is not that I wanted XP to be an overblown process, specifying roles and practices for everything related to development. But, for example, there is an assumption that the customer just knows how to generate good stories (those with high business value) in XP. The customer creates stories, prioritizes them, and knows when to extend, revise or remove stories from the project. There is no guidance for the customer as to how to be a "good" customer.

A thought solidified for me in a recent blog post, regarding the role of testers in XP. I realized that XP is a coding / developer centric process. Eight of its twelve practices are directly about the creation of code; the remaining four support the developers in their work. Story generation and other concerns are outside the scope of XP.

In my opinion, this is a strength, rather than a shortcoming. Software development is about producing software, so it is natural to start with a process that is about generating code. XP is lean and focused on this task; there is little fluff. It allows the project team the freedom to add complimentary processes that support other activities and roles that may become necessary or require assistance.

For example, scrum (a project management centric process) is compatible with XP. For teams that have greater project management needs (or an available project manager), scrum can be added to the development process. But if it's not needed, then there is no need to be encumbered by the extra process.

I think this encourages a "on-demand, just in time" addition of process. The advantage of this approach is that the development process is kept lean. Starting with RUP for is like managing your development process in waterfall fashion; you preemptively specify all your project's process requirements and devise a design.

Alternatively, active process refactoring, facilitated by techniques such as retrospectives, allow the development process to evolve with the needs of the team and their context. This allows a team to adhere to the YAGNI principle (You Ain't Gonna Need It) of simple and lightweight process.

Unfortunately, process refactoring does not have the benefit of automated unit test / change detection suites. In XP, such tools make refactoring safe; you get immediate feedback as to what you broke and what dependencies exist; process refactoring can be riskier.

For me, this idea feels right; it comforts and helps alleviate many of the concerns I have regarding applied agile development. I think the next step is to identify process "smells" and their possible refactorings. (I have done some work related to this. See my Implementing Agile Process wiki section or the Agile Process Pattern Catalog.)

discovery

Requirements, risks, architectures, strengths, weaknesses, proficiency, and throughput are examples of project and team elements that emerge or are discovered in an agile process.

unencumberment

Either get out of the way of X or keep Y out of your way. Remove anything that hinders your workflow and the progress and success of the project.

feedback

An agile process provides constant feedback along many dimensions of the project. This may take the form of an XP team's velocity, dependencies during refactoring from unit tests, Scrum Burndown Chart, or the frequent evaluation of software by the customer from frequent iterations.

sustainable, consistent pace

The month sprint of scrum, 40-hour XP workweek are examples.

synergy

The practices support each other. For example, collective code ownership and pair programming support each other in XP.

Besides the "by definition" consideration, existing agile processes, such as XP and Scrum, fit the above guidelines. I have not used Mary Poppendieck's Agile Customer Toolkit or Lean Software Development Toolkit, nor have I been part of a Feature Driven Development project, though I think they also conform to my guidelines.

Furthermore, I think the agility of an agile process is a measure of how the process contributes to the efficiency and productivity of the project and team. This contribution doesn't imply success, but should promote speed and responsiveness.

Unfortunately, I think it is very hard to measure agility; it is inherently qualitative and context biased . For example, I have always felt that Feature Driven Development isn't as agile as XP. I think the reason is one of encumberment. The documentation and project mangement aspects of FDD seem to encumber the project. However, you can make the argument that FDD may better promote the process of discovery and management, from the customer's perspective. XP provides little guidance for the customer as to the discovery of stories that provide the greatest business value.

I feel these guidelines offer a different perspective than the elements of the manifesto. For example, communication and collaboration are desirable because they promote discovery and provide feedback. As I consider the experiences I would characterize as "agile", I am better able to articulate their "agility" in terms of these guidelines.

Armed with this idea, I will now reconsider my thoughts on such questions as "what is Agile Testing?"

I have been spending a lot of time thinking about testers and XP lately. It has caused me to start thinking about other typical roles and their place in XP. So, what about software architects and XP?

The short answer is that they have no place. I say this with some confidence, as I used to consider myself a software architect. (Over the last few years I have become less and less comfortable with associating myself with the moniker.)

The incompatibility comes from the basic assumption software architecture makes: you can successfully design an architecture for a system in upfront fashion. Besides being in direct contradiction to the practices and principles of XP, it is practically flawed. Software needs to change. In my experience, most people are very hesitant to change that which they have made large upfront investments in. I've seen (and at times unfortunately architected) systems where the architecture became an impediment; it had to be overcome and accommodated as the system evolved.

This doesn't mean software architects should be fired from an XP project. They will, however, have to change their perspective and work habits. On an XP team, they will be "just" another developer. Their architecture expertise will be utilized as they pair with other developers.

In my opinion this is actually liberating for the whole team. First, there isn't the elitist notion of hierarchal roles on the team. Furthermore, as with a technical lead, there is an assumption and pressure that the architect knows all the answers. In XP, this myth doesn't have to be maintained; the architect can now offer their advice and expertise when appropriate and defer to that of others when appropriate. In XP the architecture evolves and is owned by the team, not one person or group.

I can sympathize with the allure of software architecture. It can be very satisfying to create an elegant design solution for software. But it is also frustrating and stressful to watch the once-pristine architecture get band-aided and fail as needs change. Ultimately, I am a pragmatist. I would much rather construct working software than pretty diagrams.