Christian Sepulveda's Blog Archives

December 11, 2003
XP and Customer Tests: Is it fair?

Maybe the customer shouldn't write the acceptance tests, or at least all of the acceptance tests. I know this sounds like heresy, especially coming from an XP coach. But let me explain...

I'd like the customer to write the acceptance tests. I think they must be intimately involved in the acceptance testing of an application, as the customer is determing the features. But, within the XP literature and discussion groups, there is an emphasis on the customer's providing the acceptance tests. Work cannot and should not begin until this happens. If the customer can't define acceptance criteria for a feature, how can she expect a developer to fulfill her expecations.

I think there is a small contradiction here. One reason for small iterations, an important agile and XP practice, is that it provides feedback, mitigating the "I'll know what I want when I see it" syndrome. So, if the customer is inexperienced playing the role of an XP customer, she may not know how to provide acceptance tests.

Consider a developer who is new to test driven development. In my experience, learning how to precisely define the expectations of functionality before you write the code can be a challenge. How do you test the layout of widgets on user interface? Should I test color and font? An XP coach know he must mentor developers when learning TDD. No one expects a developer to magically adopt this skill.

So why do we expect the customer to magically have the ability to write acceptance tests? At least developers are trained in logical and cognitive activities; the customer may not be.

Some customers are naturally adept at writing acceptance tests. Such a team can only benefit from such a customer. But what about the others? I think they should be trained, mentored, assisted and extended the same patience a developer would be when learning TDD. The feedback and experience of completed iterations should dvelop the customer's skill at generating acceptance tests.

There is another reason I don't think it is fair to expect the customer to be soley responsible to write the acceptance tests. Frequently, the domain logic of a system has complex boundary cases and permutations that require careful analysis. Not all customers are comfortable or capable of this analysis. Developers could add a lot of support to the quality of the project, if they view this as their responsibility. (A skilled tester would probably be even better, but this is a different story.) I think the notion of an cohesive, collaborative team means all team members should support the goals and results of the project.

In practice, I know most XP teams and coaches will support the customer and acceptance tests when necessary. I am not proposing an excuse that allows the customer to shirk any responsibility. But I preceive a disconnect in the published attitutude of the XP community and reality on this topic.

Posted by csepulv at 11:54 PM