Christian Sepulveda's Blog

November 16, 2005
Agile 2.0 and Web 2.0: A Perfect Match

Okay, so my title may be little exaggerated, especially since there isn't officially an Agile 2.0. However, for the agile development community, there are many shifts, trends and new ideas that make it feel like a new generation of agile development. (More on this later.)

Anyway, I've been following some of the Web 2.0 developments lately; I've read many blog posts (http://web20workgroup.com is a good place to start for those interested), heard gossip about VC funding, and know a few people working on Web 2.0 projects. All of this information makes me think an agile approach is perfectly matched for the development of Web 2.0 applications.

Agile development emphasizes the need for feedback, discovery and adaptation. A common agile credo is "release early, release often." Agile development is basically a genetic algorithm (for more on this, read this post); an agile approach evolves its software. Evaluating the "fitness" of the application with each iteration is critical. Real user feedback is one of the best indicators for making such an evaluation.

Most Web 2.0 projects (or at least those I have seen) are consumer oriented. Consumer oriented, service-based software markets are generally accommodating (and sometimes welcome) frequent releases, assuming each release is at least as good and as stable as the previous. Agile development favors short iterations and core practices such as automated testing and continuous integration develop the ability to release early, often and reliably as a core competency of its adopters.

Most projects in early stages are based on a vision and skeleton of a possible solution; success is largely determined by how quickly the application can evolve into what user's need. Quick delivery is necessary to compete and not miss windows of opportunity. I think the best software process to support this is based on an agile model. One of my former clients (when I was a consultant) believed this as well and feels that their agile adoption was a key facilitator of their IPO last year. I also know some very large, high profile, Bay Area web companies who are adopting agile software development and see it as necessary to remain competitive.

I think others support the complementary nature of agile and Web 2.0; the canonical book on Ruby on Rails, one of the popular Web 2.0 enabling technologies, is Agile Web Development with Rails. It describes how Rails development embodies and accommodates an agile approach.

I noted that I believe the agile community is undergoing Agile 2.0. Last year the 2nd edition of the XP white book was published, in which Kent Beck revised and updated his description of XP. Mary Poppendieck's keynote address at XP/AU 2004 described agile as "crossing Moore's chasm"; it is shifting from early adopters to mainstream. While some feel this shift may water down agile, I think it can amplify its effectiveness; many in the agile community have updated and evolved their ideas to better support the software development process.

Finally, I have also adopted a more mature understanding of agile development. I've had many successes in my career applying agile methods, but have been challenged in my current job and our agile adoption. These challenges have caused me to reflect on many assumptions and ask many questions. Though I have more questions than resolutions (which I think is a good sign you are actually learning), my understanding and application of agile software development has evolved into its own "next generation." I know a few others who share many of my ideas and are working in the Web 2.0 space. For these instances, I am excited to see how the combination of Agile and Web 2.0 turns out.

Posted by csepulv at 02:18 PM | Comments (0)


November 11, 2005
Fitness Functions and Agile Development

Agile development is structured on iterative and adaptive development. The application evolves over time and, at least hopefully, closer matches the user's needs. The basic process of agile development is a genetic algorithm; after each iteration, adaptations are made and the whole process repeats with each iteration.

The success of a genetic algorithm generally depends on the quality of its fitness function. A fitness function evaluates the success, or "fitness", of a given result. Within agile development, the iteration boundary marks an opportunity to evaluate product backlogs, prioritize features, and generally tweak the process; both the product and the process evolve.

In order to have a successful agile project, it is imperative to have fitness functions for evaluating both the application being developed and the process by which it is developed. The agile community provides various guidelines for the process evaluation; retrospectives, cost of change, quality of team dynamics, test suite run time and size, shrinking code bases, etc. are all assessment tools and indicators of process health. However, there is little information about evaluating product health.

There are some obvious indicators and questions: do users like it, are bug counts low, is the product selling? I think these are symptoms of the relative health of the application, but I think you need more from a suitable fitness function, particularly when the product isn't an instant and obvious success.

For example, prioritizing features can be a very difficult activity; users will typically assign many features a "number 1" priority. However, ranking each feature in strict order, with no duplicate priority, is a challenge.

While the assessment of the product's health can be seen as a marketing or product management activity, I do not think it is outside the scope of agile development. Successful projects depend on the collaboration of cross-functional teams. Thus, I think it is important to communicate the fitness criteria for an application amongst all on the team, even if one group has primary ownership of it.

Ultimately, the fitness function for an application is an answer to the question of "why do we build the products we do?" Do you know the answer to this question for your current project? Ask others and you may be surprised at the responses.

Posted by csepulv at 02:58 PM | Comments (0)


Agile, Waterfall and Core Assumptions

Superficially, there are many ideas and practices that are in common with agile development and waterfall development. There is a Agile Unified Process, for example and an "XP plugin" for the Rational Unified Process. However, I think agile development is quite different than waterfall, though it is not always easy to name the differences and understand why drawing strong distinctions matters.

I think agile and waterfall are each based on a fundamental assumption; if you subscribe to this idea then you probably should adopt that process. The assumption is about how to achieve a successful software project.

For waterfall development, I think the basic assumption is that perfect planning leads to optimal results. Agile assumes is that an adaptive and evolving approach is the way to go.

I do not think anyone who adopts waterfall actually thinks perfection is attainable, but I think the entire process is aimed at the pursuit of a perfect plan. Extensive requirements, complete with test traceability, elaborate architectures, long analysis and design phases, and up-front test plans are attempts at precise, accurate plans. Usability, overcoming technical challenges and meeting deadlines are achieved (or so is the idea) by building a plan that satisfies these requirements.

By contrast, agile development tends to be lightweight on planning. There is a general pragmatism to "do as little as you can get away with" when considering planning, requirements capture, architecture and documentation, as emphasizing planning would emphasize a different base assumption. However, agile development, in particular XP, is very heavy weight when it comes to automated testing; tremendous effort is usually expended. The reason is the emphasis on adaptability and evolution. Agilists assume change is a necessary and inevitable and agile processes encourage developing your "accommodate-change muscles." Thus, automated tests tend to be a primary mechanism to achieve efficient regression testing, which accommodates code design changes, for example.

Anyone who reads my blog knows I am very biased towards agile development. Though it may sound arrogant and obnoxious, I think many of the practices of waterfall are based on myths and thus make the process flawed. For example, the practice of software architecture is the application of learned patterns and principles in an attempt to satisfy functional and non-functional requirements. The problem is that technology changes quite fast and the software industry is too immature; it is hard to successful apply experience when the rules, constraints, and contexts are in flux. For at least the foreseeable future, I think you are better served with a process that allows for discovery and adaptation than depends on rare constants.

Posted by csepulv at 01:57 PM | Comments (0)


September 23, 2005
Guidelines for Being a Strong Job Candidate

I have interviewed a lot of people over the years. (I've hired over 50 people, interviewed hundreds, phone screened many more and read thousands of resumes.) While I don't claim to be any sort of recruiting expert, there are two things I can intelligently (as much as I can for anything else) speak of: trends in resumes / candidates and guidelines for being stronger on both.

The single most important fact to understand as a job seeker is: A prospective employer has no attention span whatsoever.

Your goal is to get the attention of the person who will actually decide to hire you and make clear why you are different from all the other candidates she comes across. Employers must sift through many resumes (there were times I would look at 100 resumes a day) and frequently cannot afford to spend a lot of time on a resume. Assume you must communicate in 10 seconds a reason that the employer should continue reading. Recruiters, internal or external, are not your target audience. While you may need get past them to reach the actual employer, never forget that your message should be targeting the actual employer.

While this sounds obvious, you are selling yourself to an employer. However, most resumes I see are not selling but reporting. Listing the details of each job, tool, language, API, or activity from your entire experience is not effective.

A general sales adage is a product should be made easy to buy, not simply easy to sell. When faced with buying something that is a "no-brainer", there is no selling. A candidate should always consider the perspective of the employer; what would make it a "no-brainer" to hire me?

There are many different approaches and considerations for how a candidate can position herself as an obvious choice. Ultimately, I think it can be summarized by two questions (from the point of view of the employer): What can this candidate do for me? How does this help me? The first question is about the results the candidate can achieve or deliver. The second is about the match of these skills for the employer's context.

The following are some guidelines for candidates:

Overall

Have a 30 second elevator pitch.
Know what your strengths are. Know what results you have achieved. What makes you special and stand out?
Know your professional goals.
What are your career plans? Where do you want to be in one year? Five years?
Have a career plan.
The goals are a strategic concern; your plan is tactical. You should be able to describe the role you want to perform, with detailed functions and responsibilities. Be able to describe an ideal work day, month, or year.
Know your strengths, weaknesses and what affects them.
More importantly than what you do well or poorly are the factors that enhance your skill set and those that detract from it. These are critical to selecting the right position.

Resume

A resume makes the first impression. It must get someone's attention immediately and should answer the two most important questions above for the employer.

1 page only. No exceptions.
I don't read 7 pages resumes; if the person can't be brief, it implies they are disorganized and cluttered. I will note one caveat to this: your resume should be one page. It is an introduction and summary. If there are details that are important to communicate, include an addendum or appendix (which should be never more than 2-3 pages) and make it clear that this is extra, not part of the resume. You can title it "Project Details" or something, but it should have it's own header and not be confused as part of the resume.
Lead with the elevator pitch.
Your 30 second elevator pitch should be communicated immediately in a "Professional Profile" or similar section.
Focus on results and achievements
When I read that a candidate was on a project that tripled company revenues, I am interested to continue reading.
Highlight technical qualifications
Do not list every three letter acronym you know, even if you are an expert. Focus on your core strengths and the technologies you want to work with or that are relevant to the employer. If you know C++ but are looking for job doing Java, don't mention you know C++.

Almost 100% of the resumes I read look the same. They all share the same subset of technical jargon references and technologies. There are many, many J2EE, .NET, C++, etc. developers. It is almost impossible to differentiate yourself in this category. Though a technical profile is obligatory, as a technical worker, it should be brief and contain highlights. I suggest putting it at the end of the resume.

The technical profile of a resume is the source of the largest mistakes in candidate positioning, so I will babble and rant about this a bit. I think the reason for these mistakes are:

  • technical people focus on technology
  • many recruiters encourage listing all your technical experience
  • employers may screen based on the content of technical profile

The first point is probably pretty obvious and I won't discuss it. The second is also common, though I discourage it. The last reason is the most significant: frequently resumes are so poor (or there are so many) that employers have little choice than to screen based on technical profiles. However, I think there are a few mistakes being made here. First, candidates try to keep as many options open as possible and this actually hurts them. Few people are experts in the large lists of technologies, languages and tools commonly listed on resumes. Even if you are, simplify. For example, if you've worked with many Web Service / SOAP technologies, write in your technical profile something like "SOAP / Web Services expert (worked with Axis, WebMethods, BlueTitan ...) instead of listing the 20 different Web Service related frameworks you know. Alternatively, if your goal is to work on web services, only list the web service frameworks you know and omit the other items such as object relational mappers, databases, etc.

Listing too many elements in a technical profile has the risk of implying indecision and insecurity. Don't list technologies you don't really want to work with. Also, a candidate that doesn't care what technologies they work with (when they are listing many, unrelated technologies) seems to have little career focus. Finally, it may convey insecurity in the candidate; it can make a candidate look desperate and thus undesirable. Less is more.

Phone Screens

In over 90% of the phone screens that I do, I know if I will pursue a candidate in the first two minutes of the phone screen. A phone screen is intended to serve answer one question: Should I invest the time to formally interview this candidate? Formal interviews are generally very expensive to conduct and therefore most organizations don't do them lightly. There are a few things to keep in mind at this point.

Research the company
I hope this is obvious, but it amazes me how many people do not do this. Look at the companies website, competitor websites and industry websites for the employer. Learn as much as you can.
Be prepared
Know your own experience well, such that you can communicate succinctly. Learn how to be brief and focus on highlights. Interviewers will ask follow-up questions when interested and will appreciate brevity. Also, give thought to typical questions and know your answers. Such questions include: Why are you looking for a job? What is the ideal job, role, organization? What is the most important thing I should know about you as a candidate? Tell me about an interesting technical or team challenge and how you resolved it. What was the best project your were on? Worst? Why?
Interview the employer organization
You should look as closely at the employer company as they are looking at you. Good candidates take their professional development very seriously and are engaged in knowing about the company, its culture, its business model, future, history, etc.

If you are looking to work for me...

This entire blog entry is biased about making a candidate attractive to me, as an employer, but there are a few other items of note:

Communication Skills are Key
Being able to communicate clearly, effectively and succinctly is critical. It is vital to working in an agile environment and poor communication is one of the core reasons I don't pursue candidates.
Must be a Team Player
Agile Processes emphasize collaboration. You must thrive in a team environment.
Be Honest
I generally know when a candidate is weak in a particularly area, unsure or blatantly lying. I prefer and respect an honest and direct "I don't know" rather than trying to dance around ignorance.
Be Committed to Projects
I am looking for people who will take ownership of the result and do what they can to achieve it. I want candidates who take the outcome as a personal reflection of their abilities and work ethic. I don't ask for late nights or weekends but respect people who sacrifice when they need to make sure something is done.

Conclusion

Employers hire people, not cogs. A candidate should never forget that an employer is investing in a person and should do everything they can to position their value as an individual to an organization. If you are seen as just a Java developer who knows SOAP, you are easily replaced. Great employees, however, are assets to companies and rare.

Posted by csepulv at 05:53 PM | Comments (6)


September 02, 2005
The Importance of Iterations in Bridging the Market / Application Gap

In a recent blog post I discussed the "Market / Application Gap." In order for a software product to succeed, it must align with the user's needs. The faster an organization can bridge the gap between the market and the actual application, the greater its competitive advantage and its opportunity for success.

At the start of a project, some effort is given to requirements discovery and this effort varies with software methodology. From here requirements are communicated to engineers who in turn produce an application that reflects the requirement. With agile software development there is an emphasis on taking action as early as possible; waterfall tends to focus on a more complete sets of requirements, designs and test plans (read: more time). With my "Market / Application Gap" diagram in mind, the process looks something like this (order of events, with respect to information flow / deliverables, are numbered):

This cycle represents an iteration: from requirements to delivery of an application. While much has been discussed about the iterative nature of agile development, the truth is that every software process uses iterations. The only question is the length of the iteration. An agile project might have an iteration length of weeks to months; waterfall can have iterations that last years. (Remember, I am defining an iteration in this context as the delivery of an application to the end user.)

At the end of an iteration, the overall gap between the market and the application either grows or shrinks. If you have an iteration cycle of a few months (at most), you can recover if you missed the needs of your market. When you have an iteration cycle of years, you may not get a second chance if you missed the mark. This is one of the core reasons why agile development makes more sense than waterfall: from a risk management point of view it is foolish to take a "big bang" approach where you have few opportunities to satisfy your market.

If the goal is to close the gap between the related participants in the "Market / Application Gap", the quicker you can complete an iteration, the quicker you possibly close the gap. Many successful software products simply work the way users expect; unless you're omnipotent, this requires some trial and error and probably some mistakes. Users will tolerate some mistakes, but won't wait around long. Long iterations can result in a lost audience, which spells doom for commercial software. In my years of software development, I've seen a variety of software processes, many of which claim to encourage iterations and quick feedback. Agile Software Development is the only approach I've seen deliver on this promise in actual practice.

Posted by csepulv at 03:26 PM | Comments (0)


August 15, 2005
Lowering the Cost of Change

Lowering the cost of change is a frequently cited benefit of adopting agile development and extreme programming, in particular. You'll frequently see graphs drawn that show an exponentially growing cost curve as the cost of change for a waterfall project and graphs that show a level or even down sloping (after an initial ramp up cost) cost curve for XP projects.

I personally have been in involved in projects where the down sloping cost curve was actually achieved, so I know it is possible. I also know it is rare; on other projects a flat cost curve was all that was achieved. Both are good outcomes, but what is the cause for the differences in these projects? How does a project actually achieve low costs of change?

However, before exploring how to lower the cost of change, defining the cost of change is necessary. I frequently see discussions about the cost of change that center around the time or effort required to implement a story. While not wrong, I think this perspective is too narrow and short sighted. This definition is not affected by a team that is "going in circles". But who cares if they can go in circles really fast?

Mary Poppendieck has discussed a Maturity Model of Software Development. (She has a presentation on this and an article published in Software Development magazine. http://poppendieck.com/maturity.htm) The basic idea is that what matters is how quickly and reliably can an organization take a user request and deliver it in the software. This is the cost of change I wish to focus on: the amount of time elapsed and resources spent on taking a user feature and having it present in the software application.

Now, returning to analyzing the cost of change itself, there are two considerations: what contributes to the cost of change and how does agile development help lower it.

Contributors to the cost of change:

  • time to assess and prioritize a user request
  • overhead for specifying a story
  • "churn" in stories
  • number of changes to code base to implement story
  • cost of release

Time to Assess and Prioritize a User Request

How is feedback handled? How many people are part of the decision? The longer it takes for a sales person to communicate a user requirement to product management, have it prioritized and then worked on by engineering, the hight the cost of change. Similarly, the number of people (or worse yet, committees) that must approve or deny the request, the higher the cost of change.

Overhead for Specifying Stories

The time and number of people it takes to specify a story and complete set of acceptance tests affect the cost of change.

Story Churn

The higher the number of times a feature area is revisited, be it because of bugs or revisions due to what was delivered wasn't quite what user's wanted, the higher the cost of change.

Code Base Changes Required to Implement a Story

The time it takes to assess and discover the necessary code changes and to implement them impacts the cost of change.

Release Cost

What is the overhead to a release? Is there a manual QA cycle? How long does it take? Documentation done at the end?

How can Agile lower the Cost of Change?

The primary mechanism for lowering the cost of change is bridging the gaps between the user, product owner, developer and application. (For more on this, please see this blog post.)As the gaps are closed, the application tends to be in alignment with the user requirements.

Practices such as an onsite customer and acceptance tests help develop a shorthand of communication between the product owner and the developers. Thus, the overhead of specifying stories tends to go down and existing acceptance tests and infrastructure serve as templates and extension bases for new acceptance tests.

Iterations provide many benefits. They allow opportunities for injecting feature requests, shortening the time between a user's initial request and when she sees it in the product. Iterations, combined with engaging a cross functional team in each iteration, keeps activities such as documentation and exploratory testing in step with development, shortening release cycles.

Finally, the engineering centric nature of XP encourages the application, and more importantly the underlying code base, to become ever closer to expressing the user's intentions and needs. It is very gratifying as a developer when your code base almost anticipates new user features.

I think software teams should actively try to measure, monitor and lower the cost of change. The complete "cost chain", from sales to development, needs to be included and managed. Agile practices, such as XP, will provide a lot of guidance for improving the elements closest to the code, while applying general agile principles, such as cross functional teams, iterations, high communication, emphasis on feedback, can help the other aspects not directly involved with development. An organization that can successfully manage and lower the cost of change will achieve significant competitive advantages such as lower cost and time to market. Organizations that can't manage the cost of change will have trouble competing at all.

Posted by csepulv at 10:35 PM | Comments (0)


August 09, 2005
The Importance of Closing the Gap Between Customers and Developers

I think in a software project there are varying notions of reality regarding the actual application being developed. They are:

  • actual needs of the market or user base
  • product owner's vision of the application
  • developer understand of the goals and requirements of the application
  • code base and actual executable application

This is represented in the following diagram:

The gap between these perspectives is normal, particularly at the start of a project. However, I think it is difficult for a project to succeed if the gap is not being closed; there needs to be alignment and overlap among the different viewpoints.

One of the motivations of agile software development, at least in my opinion, is to help close these gaps. The high bandwidth of communication, intense collaboration, emphasis on feedback and the importance of acceptance tests are techniques to communicate the assumptions, needs and current state of each different constituent. With each iteration, the gap should be bridged a little more.

This raises an interesting question: How can you measure or monitor the shrinking (or worse the expanding) of the gap?

I don't think you can measure this directly. However, there are a variety of indicators that can help evaluate how well you are closing the gap.

Bugs are probably the best indicator. A bug represents a mismatch in expectations, understanding or communication. Bugs are reported when a calculation is incorrect as well as when a user doesn't like the number clicks it takes to perform an operation. In both cases, there are one or gaps between what the user needs and what the application is providing; perhaps the product owner specified a requirement that made the user experience awkward or a developer missed a test case. If your bug rates aren't going down it is an indicator that the gaps are not shrinking. I suggest assessing the cause for the constant (or growing) bug rate by evaluating which gap is the widest. Understanding where the communication / comprehension breakdown exists is the most important step in resolving it.

There are other indicators as well, such as the amount of churn in stories (requirements) and the overhead of specifying stories. If you are revisiting the same areas of the application frequently, whether the reason is user dissatisfaction or defects, there are gaps between the user and the product owner or between the developer, user and code base.

There are ultimately two reasons for closing the gap: delivering a product that users like (and will buy) and lowering the cost of change. One reason is a huge component of succeeding today, the other allows you to succeed tomorrow. When the gaps are small and there is alignment between the user, the application and all those in between, you will probably have a winning project.

Posted by csepulv at 12:55 PM | Comments (0)


Cleaning out the Cobwebs...

My blog has not seen a post in a long, long time... My last post even promised that I would post more.

Excuses aside, I do sincerely plan a lot more posts in the coming future. As I noted in an earlier post, I have taken a position at one of my clients, Nominum. I've been an employee a bit over a year and it has been a very interesting experience.

I've been ranting for a while that XP is an engineering practice and that there is more to successful software projects than what agile development offers. I've been critical about the silence (in my opinion) of the agile community and literature on the missing elements that actually have the largest impact on a successful project. I had theories (some at least partially verified in my experience) about the interdependencies between the various elements of the "value chain" in a software project.

My experience at Nominum has smacked me in the face with all the things I've ranted about. Its confirmed many ideas, but also posed many challenges. It is still a work in progress, but I revisted and revised many of my ideas regarding XP and agile software development. I plan to post about them in the near future.

Posted by csepulv at 12:53 PM | Comments (0)


March 13, 2004
TDD: Maybe "tests" is the wrong word?

There have been many discussions about the appropriateness of the term "tests" in describing the unit tests created during test driven development. Most of the debate has centered around the overloaded use of the term "test". For many, the expectation of testing is to discover bugs. There are also many forms of testing, such as usability testing and exploratory testing. Generally, I think the traditional role of a tester (and most forms of testing) are about the act of discovery.

Test Driven Development is more about verification than discovery and I think this is one core motivation of TDD. In TDD, you specify your expectation first, then write code to satisfy it; the test verifies this contract.

Though I am not making any grand revelation, I think there is a subtle importance in the distinction I've made. I was recently asked about test coverage, completeness and TDD. There has been a lot of interest and research regarding test coverage and completeness. There have even been attempts to provide formal proofs of correctness for software programs. The desire is to have a verification mechanism. TDD does provide good test coverage of code bases, though it doesn't guarantee there will be no bugs or other unexpected behavior. TDD simply provides a mechanism for verifying that declared expectations have been satisfied.

Since I think testing is more about discovery, I think it is a very different type of activity. One of the challenges of testing is that you never know when you are done. Therefore, it is somewhat pointless to ask about test coverage or completeness. How would you define "complete"? The act of testing is about the discovery and refinement of expectations. As expectations are discovered, a verification can be captured, satisfied and automated.

There has been a lot of debate regarding the need of manual testing compared to automated tests. For me, if I use the moniker verification, there is no debate. Verification operations should be automated; testing should not.

Posted by csepulv at 10:20 PM


January 12, 2004
The Cost of NOT doing TDD

This is a revised posting. I got a little carried away with my math in the example. Sorry.

I was working with a team that had been gradually adopting agile practices, but the developers were resistant to practicing Test Driven Development. At one point, I decided to start tracking how much time was spent compiling code, running the application/debugger, re-establishing the context in the application, observing a result and then closing the application. The developer would think about what he observed, make code edits, start the debugger and repeat the cycle. I observed that it took four minutes on average to run the cycle of compile, launch debugger, find context, etc. This would be done an average of eight times per hour; the remaining hour was spent thinking and making edits.

One half of each day was completely wasted. The developer couldn't really doing anything productive during the four-minute debug cycle because he had to navigate to the corect place in the application and execute the right steps so he could observe the results of the change he just made; he was forced into tedium. In other words, half the development budget was being spent on performing repeated, monotonous tasks. This isn't a very effective use of resources or talent.

The development world has been doing this for years. My analysis may seem a bit naive; the lost time I refer to has been considered a necessary overhead in order to receive feedback. The feedback is necessary but this method of getting feedback is not.

Consider the use of TDD. The same developer makes a code edit, runs an automated test suite and in seconds gets feedback of the impact of his change. Besides knowing the expected impact, the developer knows the impact of his change on the system as a whole, performing a validation of his expectation and regression check in one step. (My analysis of the four-minute debug feedback cycle didn't address the time spent determing the system impact of code changes).

Faced with such results, the team in my example did adopt TDD. Their feedback loop was reduced to thirty seconds. In the example, there were eight edit cycles per hour. Previously, thirty two minutes were being used to get feedbak regarding an edit, so twenty eight minutes were being used to make the actual change, consider options, checkin code, etc. This is roughly three and half minutes per edit, for eight edits per hour.

So now, an edit cycle costs a total of four minutes, which doubles the number of edit opportunites. The greater the number of opportunities you have to introduce edits, the faster your progress. This, in my opinion, is one of the reasons teams using TDD go faster. They get similar feedback they would have received in a debug cycle, but it is compressed so much that it empowers the team. The team has so many opportunities to try something, they can afford to experiment with design changes, clean up code or simply move forward. Not only will the team go faster, but the resulting code is generally more flexible, resusable and readable. (I haven't enough discussed the benefits of regression testing and its related cost savings.)

So the when faced with people resistant to TDD, maybe the question isn't why to use TDD, but how can you afford not to?

Posted by csepulv at 12:46 PM


December 11, 2003
XP and Customer Tests: Is it fair?

Maybe the customer shouldn't write the acceptance tests, or at least all of the acceptance tests. I know this sounds like heresy, especially coming from an XP coach. But let me explain...

I'd like the customer to write the acceptance tests. I think they must be intimately involved in the acceptance testing of an application, as the customer is determing the features. But, within the XP literature and discussion groups, there is an emphasis on the customer's providing the acceptance tests. Work cannot and should not begin until this happens. If the customer can't define acceptance criteria for a feature, how can she expect a developer to fulfill her expecations.

I think there is a small contradiction here. One reason for small iterations, an important agile and XP practice, is that it provides feedback, mitigating the "I'll know what I want when I see it" syndrome. So, if the customer is inexperienced playing the role of an XP customer, she may not know how to provide acceptance tests.

Consider a developer who is new to test driven development. In my experience, learning how to precisely define the expectations of functionality before you write the code can be a challenge. How do you test the layout of widgets on user interface? Should I test color and font? An XP coach know he must mentor developers when learning TDD. No one expects a developer to magically adopt this skill.

So why do we expect the customer to magically have the ability to write acceptance tests? At least developers are trained in logical and cognitive activities; the customer may not be.

Some customers are naturally adept at writing acceptance tests. Such a team can only benefit from such a customer. But what about the others? I think they should be trained, mentored, assisted and extended the same patience a developer would be when learning TDD. The feedback and experience of completed iterations should dvelop the customer's skill at generating acceptance tests.

There is another reason I don't think it is fair to expect the customer to be soley responsible to write the acceptance tests. Frequently, the domain logic of a system has complex boundary cases and permutations that require careful analysis. Not all customers are comfortable or capable of this analysis. Developers could add a lot of support to the quality of the project, if they view this as their responsibility. (A skilled tester would probably be even better, but this is a different story.) I think the notion of an cohesive, collaborative team means all team members should support the goals and results of the project.

In practice, I know most XP teams and coaches will support the customer and acceptance tests when necessary. I am not proposing an excuse that allows the customer to shirk any responsibility. But I preceive a disconnect in the published attitutude of the XP community and reality on this topic.

Posted by csepulv at 11:54 PM


XP and Customer Tests: Is it fair?

Maybe the customer shouldn't write the acceptance tests, or at least all of the acceptance tests. I know this sounds like heresy, especially coming from an XP coach. But let me explain...

I'd like the customer to write the acceptance tests. I think they must be intimately involved in the acceptance testing of an application, as the customer is determing the features. But, within the XP literature and discussion groups, there is an emphasis on the customer's providing the acceptance tests. Work cannot and should not begin until this happens. If the customer can't define acceptance criteria for a feature, how can she expect a developer to fulfill her expecations.

I think there is a small contradiction here. One reason for small iterations, an important agile and XP practice, is that it provides feedback, mitigating the "I'll know what I want when I see it" syndrome. So, if the customer is inexperienced playing the role of an XP customer, she may not know how to provide acceptance tests.

Consider a developer who is new to test driven development. In my experience, learning how to precisely define the expectations of functionality before you write the code can be a challenge. How do you test the layout of widgets on user interface? Should I test color and font? An XP coach know he must mentor developers when learning TDD. No one expects a developer to magically adopt this skill.

So why do we expect the customer to magically have the ability to write acceptance tests? At least developers are trained in logical and cognitive activities; the customer may not be.

Some customers are naturally adept at writing acceptance tests. Such a team can only benefit from such a customer. But what about the others? I think they should be trained, mentored, assisted and extended the same patience a developer would be when learning TDD. The feedback and experience of completed iterations should dvelop the customer's skill at generating acceptance tests.

There is another reason I don't think it is fair to expect the customer to be soley responsible to write the acceptance tests. Frequently, the domain logic of a system has complex boundary cases and permutations that require careful analysis. Not all customers are comfortable or capable of this analysis. Developers could add a lot of support to the quality of the project, if they view this as their responsibility. (A skilled tester would probably be even better, but this is a different story.) I think the notion of an cohesive, collaborative team means all team members should support the goals and results of the project.

In practice, I know most XP teams and coaches will support the customer and acceptance tests when necessary. I am not proposing an excuse that allows the customer to shirk any responsibility. But I preceive a disconnect in the published attitutude of the XP community and reality on this topic.

Posted by csepulv at 11:54 PM


November 27, 2003
Scrabble and Dummy Data Objects

Lately, I've been trying to explain the reasons why dummy-data classes are a smell in software. Dummay Data classes are classes that do not have behavior, only data.

I thought of a different way to illustrate the problem, as the common arguments have not seemed sufficient lately.

Consider scrabble. When someone presents a word that is challenged, one of the first comments made is "Use 'fazooloo' in a sentence." A response such as "I was playing scrabble and I was asked to use 'fazooloo' in a sentence" is generally not appropriate. Besides being sarcastic, the sentence doesn't convey anything about the meaning of the word. The questioned term was the object of the sentence and performed no action of its own. You could substitute any term and the sentence does not lose any meaning, but doesn't improve either.

A dummy data class is like a term that can only be used as an object in a sentence. If no sentence exists in which the dummy data class performs the action, it has no purpose but to be manipulated by others. This has various implications, such as encapsulation violations, but I think the sentence analogy simplifies the reaons to avoid dummy data classes.

Martin Fowler has a bliki post about Anemic Domain Models. One characteristic that makes a domain model anemic is the lack of behavior in the objects. Furthermore, I think the domain objects must have domain behavior.

This distinction can have subtle design implications. I worked on a system where we had an elabrorate domain model (or so we thought), but the classes where really dummy data classes. When I pointed this out, others on the project claimed that since the objects knew how to persist and load themselves from the database, that this was their behavior. But from the domain perspective, the objects did nothing. Furthermore, if you then look at the implemented design, there were other smells such as violdation of the Single Responsibility Principle, as the objects where part domain, part technical infrastructure. The design of the system would be greatly improved by starting with the elimination of dummy data classes and following your nose from there.

So, when designing your classes and their collaborations, make sure each class passes the Scrabble Test.

Posted by csepulv at 11:55 AM


October 15, 2003
Agile Coaching: Some Practical Guidelines

Currently, I am working with a team that is transitioning to XP. Besides coaching the team, I am coaching one individual to be a coach so he can eventually be the team's coach. As a result, I have been thinking a lot about the nature of coaching. In an earlier blog post, I noted some of my thoughts on the abstract and theoretical components of coaching. Here, I am sharing (for what its worth) some practical guidelines for an agile coach.

Identify Expectations

Start by identifying stakeholders. I consider a stakeholder to be anyone that has an interest in the outcome of the project or will be a participant. I include developers, managers, testers, marketing staff, technical writers, and others. For each member of this list, I try to identify what are the expectations and goals of the member (or group). I prioritize the expectations and assign a weighting. I actually treat the expectations as XP stories. I will use 3x5 cards; I track the member or group that "sponsored" the story, its weighting (or cost), and priority. (I know there are other techniques, such as some multi-letter acronym matrix; these alternates always struck me as too complicated).

Take an Agile Approach

Without getting too metaphysical, I take an agile approach to coaching. (See this blog post on qualifications of an agile process.)

feedback

Retrospectives are an invaluable feedback mechanism. They should be done at the end of each iteration and the feedback received should be enacted upon.

Johanna Rothman suggests one-on-one meetings with all team members on a consistent basis. Depending on team size, this can range from every week to every few weeks, but I think the more frequent the better. Fifteen minutes to a half hour is sufficient, but the individual attention will be appreciated It provides a more secure and comfortable opportunity for feedback. I even suggest conducting these sessions offsite when possible or over a meal (or both).

discovery

Returning to the gathering of expectations and identification of stakeholders, a coach should regularly review these lists and evaluate if the expectations are being satisfied, how have they changed and who are the current stakeholders.

unencumberment

Besides the expectations of stakeholders, try to identify their primary activities and support these activities. In Scrum, the Scrum Master insulates and protects the developers so they can work efficiently. I think the coach should try to do this for the entire team. Unfortunately, this can be complicated to achieve as frequently the needs of one stakeholder are in conflict with that of another (Remember, developers and management are stakeholders). The expectation list should give some guidance for evaluating compromises and tradeoffs.

sustainable pace

Regularly assess the team's progress. When possible, try to assist, mentor or in some way help any team member that is having difficulties. Look for buy-in for the process and build on it. Hopefully these early adopters will help champion the cause as a team needs to take ownership over their process. Otherwise they won't maintain it or maximize its effectiveness.

synergy

Each of the suggestions supports each other. Retrospectives help discover expectations and needs, etc.

Ultimately the coach is part leader and part manager. In my opinion, the ideal coach doesn't have to do much as time goes on; the team chugs along by their own steam.

This is particularly fun for the XP coach as you do all this and write code too! (Context and expectations of the coach will determine the balance of activities the coach will do.)

(Ron Jefferies and Bill Wake conduct a nice tutorial on being a coach. There is a good book list that can be found at http://www.xp123.com/xplor/xp0307/index.shtml.)

Posted by csepulv at 10:21 PM


Agile Coaching: Some Practical Guidelines

Currently, I am working with a team that is transitioning to XP. Besides coaching the team, I am coaching one individual to be a coach so he can eventually be the team's coach. As a result, I have been thinking a lot about the nature of coaching. In an earlier blog post, I noted some of my thoughts on the abstract and theoretical components of coaching. Here, I am sharing (for what its worth) some practical guidelines for an agile coach.

Identify Expectations

Start by identifying stakeholders. I consider a stakeholder to be anyone that has an interest in the outcome of the project or will be a participant. I include developers, managers, testers, marketing staff, technical writers, and others. For each member of this list, I try to identify what are the expectations and goals of the member (or group). I prioritize the expectations and assign a weighting. I actually treat the expectations as XP stories. I will use 3x5 cards; I track the member or group that "sponsored" the story, its weighting (or cost), and priority. (I know there are other techniques, such as some multi-letter acronym matrix; these alternates always struck me as too complicated).

Take an Agile Approach

Without getting too metaphysical, I take an agile approach to coaching. (See this blog post on qualifications of an agile process.)

feedback

Retrospectives are an invaluable feedback mechanism. They should be done at the end of each iteration and the feedback received should be enacted upon.

Johanna Rothman suggests one-on-one meetings with all team members on a consistent basis. Depending on team size, this can range from every week to every few weeks, but I think the more frequent the better. Fifteen minutes to a half hour is sufficient, but the individual attention will be appreciated It provides a more secure and comfortable opportunity for feedback. I even suggest conducting these sessions offsite when possible or over a meal (or both).

discovery

Returning to the gathering of expectations and identification of stakeholders, a coach should regularly review these lists and evaluate if the expectations are being satisfied, how have they changed and who are the current stakeholders.

unencumberment

Besides the expectations of stakeholders, try to identify their primary activities and support these activities. In Scrum, the Scrum Master insulates and protects the developers so they can work efficiently. I think the coach should try to do this for the entire team. Unfortunately, this can be complicated to achieve as frequently the needs of one stakeholder are in conflict with that of another (Remember, developers and management are stakeholders). The expectation list should give some guidance for evaluating compromises and tradeoffs.

sustainable pace

Regularly assess the team's progress. When possible, try to assist, mentor or in some way help any team member that is having difficulties. Look for buy-in for the process and build on it. Hopefully these early adopters will help champion the cause as a team needs to take ownership over their process. Otherwise they won't maintain it or maximize its effectiveness.

synergy

Each of the suggestions supports each other. Retrospectives help discover expectations and needs, etc.

Ultimately the coach is part leader and part manager. In my opinion, the ideal coach doesn't have to do much as time goes on; the team chugs along by their own steam.

This is particularly fun for the XP coach as you do all this and write code too! (Context and expectations of the coach will determine the balance of activities the coach will do.)

(Ron Jefferies and Bill Wake conduct a nice tutorial on being a coach. There is a good book list that can be found at http://www.xp123.com/xplor/xp0307/index.shtml.)

Posted by csepulv at 10:21 PM


Agile Coaching: Some Practical Guidelines

Currently, I am working with a team that is transitioning to XP. Besides coaching the team, I am coaching one individual to be a coach so he can eventually be the team's coach. As a result, I have been thinking a lot about the nature of coaching. In an earlier blog post, I noted some of my thoughts on the abstract and theoretical components of coaching. Here, I am sharing (for what its worth) some practical guidelines for an agile coach.

Identify Expectations

Start by identifying stakeholders. I consider a stakeholder to be anyone that has an interest in the outcome of the project or will be a participant. I include developers, managers, testers, marketing staff, technical writers, and others. For each member of this list, I try to identify what are the expectations and goals of the member (or group). I prioritize the expectations and assign a weighting. I actually treat the expectations as XP stories. I will use 3x5 cards; I track the member or group that "sponsored" the story, its weighting (or cost), and priority. (I know there are other techniques, such as some multi-letter acronym matrix; these alternates always struck me as too complicated).

Take an Agile Approach

Without getting too metaphysical, I take an agile approach to coaching. (See this blog post on qualifications of an agile process.)

feedback

Retrospectives are an invaluable feedback mechanism. They should be done at the end of each iteration and the feedback received should be enacted upon.

Johanna Rothman suggests one-on-one meetings with all team members on a consistent basis. Depending on team size, this can range from every week to every few weeks, but I think the more frequent the better. Fifteen minutes to a half hour is sufficient, but the individual attention will be appreciated It provides a more secure and comfortable opportunity for feedback. I even suggest conducting these sessions offsite when possible or over a meal (or both).

discovery

Returning to the gathering of expectations and identification of stakeholders, a coach should regularly review these lists and evaluate if the expectations are being satisfied, how have they changed and who are the current stakeholders.

unencumberment

Besides the expectations of stakeholders, try to identify their primary activities and support these activities. In Scrum, the Scrum Master insulates and protects the developers so they can work efficiently. I think the coach should try to do this for the entire team. Unfortunately, this can be complicated to achieve as frequently the needs of one stakeholder are in conflict with that of another (Remember, developers and management are stakeholders). The expectation list should give some guidance for evaluating compromises and tradeoffs.

sustainable pace

Regularly assess the team's progress. When possible, try to assist, mentor or in some way help any team member that is having difficulties. Look for buy-in for the process and build on it. Hopefully these early adopters will help champion the cause as a team needs to take ownership over their process. Otherwise they won't maintain it or maximize its effectiveness.

synergy

Each of the suggestions supports each other. Retrospectives help discover expectations and needs, etc.

Ultimately the coach is part leader and part manager. In my opinion, the ideal coach doesn't have to do much as time goes on; the team chugs along by their own steam.

This is particularly fun for the XP coach as you do all this and write code too! (Context and expectations of the coach will determine the balance of activities the coach will do.)

(Ron Jefferies and Bill Wake conduct a nice tutorial on being a coach. There is a good book list that can be found at http://www.xp123.com/xplor/xp0307/index.shtml.)

Posted by csepulv at 10:21 PM


Agile Coaching: The Key is to Make Small Moves

I think the mission of an agile coach is to keep a project on the road to success. He monitors the effectiveness of the team and makes adjustments as necessary.

My father compares parenting to bumper bowling; a parent's job is to keep the ball out of the gutter. But, if their child is not approaching the edge, the parent should let the child find his own way.

Similarly, a good coach provides guidance but allows (and hopefully encourages) a team to find their own identity. It's critical for a team to take ownership of its own process if they are to maintain and adopt it. In my experience, a team will not maintain or effectively utilize an agile process, over the long term, if the coach is the only champion of the process.

Returning to the parenting analogy, the early stages of a child's development are where the parent's role is the clearest; the world is fairly black and white. But as the child develops into a teenager and young adult, the situations, decisions and consequences are more complicated.

The same is true for an agile coach. In the early stages of the coach's involvement, the team is the most willing, as they will ever be to take advice on faith. The coach can demonstrate techniques, discuss experiences and respond to questions; but the expectations of the coach are largely about process. At this time, the coach is relying on her own abilities to communicate the effectiveness and motivation of agile processes; this is the most amount of control the coach will have.

But as with the developing child, things get complicated. As the project ensues, the expectations quickly shift from process to results. Where the customer was happy to see any piece of working software in the beginning, she now has expectations of high productivity. The coach is now relying on the aptitude of the team; the coach has less direct control and must mentor the team such that they produce effectively.

As the expectations of the team increase, credibility and trust are necessary for effective coaching. Any latitude the coach will have with the team, be it management or developers, will be based on the experience of the team with the coach. It is a form of currency; successes are credits and problems are debits.

Each adjustment the coach makes is also a debit. The more direct (i.e. dictatorial or authoritative) the action, the larger the debit; subtle actions cost less. The coach has to be careful not to bankrupt herself; the skillful and wise coach judiciously spends in order to maintain reserves. A team is far more likely to trust a coach when she directly and obviously intervenes if they don't feel she is constantly crying wolf.

The master coach facilitates success by influence and suggestion. A coach shouldn't issue mandates. A good coach is a catalyst and his coaching, at times, is imperceptible. But a team usually knows when they have a good coach.

Posted by csepulv at 09:06 PM


Agile Coaching: The Key is to Make Small Moves

I think the mission of an agile coach is to keep a project on the road to success. He monitors the effectiveness of the team and makes adjustments as necessary.

My father compares parenting to bumper bowling; a parent's job is to keep the ball out of the gutter. But, if their child is not approaching the edge, the parent should let the child find his own way.

Similarly, a good coach provides guidance but allows (and hopefully encourages) a team to find their own identity. It's critical for a team to take ownership of its own process if they are to maintain and adopt it. In my experience, a team will not maintain or effectively utilize an agile process, over the long term, if the coach is the only champion of the process.

Returning to the parenting analogy, the early stages of a child's development are where the parent's role is the clearest; the world is fairly black and white. But as the child develops into a teenager and young adult, the situations, decisions and consequences are more complicated.

The same is true for an agile coach. In the early stages of the coach's involvement, the team is the most willing, as they will ever be to take advice on faith. The coach can demonstrate techniques, discuss experiences and respond to questions; but the expectations of the coach are largely about process. At this time, the coach is relying on her own abilities to communicate the effectiveness and motivation of agile processes; this is the most amount of control the coach will have.

But as with the developing child, things get complicated. As the project ensues, the expectations quickly shift from process to results. Where the customer was happy to see any piece of working software in the beginning, she now has expectations of high productivity. The coach is now relying on the aptitude of the team; the coach has less direct control and must mentor the team such that they produce effectively.

As the expectations of the team increase, credibility and trust are necessary for effective coaching. Any latitude the coach will have with the team, be it management or developers, will be based on the experience of the team with the coach. It is a form of currency; successes are credits and problems are debits.

Each adjustment the coach makes is also a debit. The more direct (i.e. dictatorial or authoritative) the action, the larger the debit; subtle actions cost less. The coach has to be careful not to bankrupt herself; the skillful and wise coach judiciously spends in order to maintain reserves. A team is far more likely to trust a coach when she directly and obviously intervenes if they don't feel she is constantly crying wolf.

The master coach facilitates success by influence and suggestion. A coach shouldn't issue mandates. A good coach is a catalyst and his coaching, at times, is imperceptible. But a team usually knows when they have a good coach.

Posted by csepulv at 09:06 PM


September 25, 2003
Agile Process Refactoring

I used to be plagued by nagging concerns about certain omissions and shortcomings of extreme programming (XP). Over recent months, I have come to the conclusion that my thinking on the topic was flawed; I used to criticize XP for not addressing a variety of issues such as the integration of testers, domain modeling, story generation, etc.

It is not that I wanted XP to be an overblown process, specifying roles and practices for everything related to development. But, for example, there is an assumption that the customer just knows how to generate good stories (those with high business value) in XP. The customer creates stories, prioritizes them, and knows when to extend, revise or remove stories from the project. There is no guidance for the customer as to how to be a "good" customer.

A thought solidified for me in a recent blog post, regarding the role of testers in XP. I realized that XP is a coding / developer centric process. Eight of its twelve practices are directly about the creation of code; the remaining four support the developers in their work. Story generation and other concerns are outside the scope of XP.

In my opinion, this is a strength, rather than a shortcoming. Software development is about producing software, so it is natural to start with a process that is about generating code. XP is lean and focused on this task; there is little fluff. It allows the project team the freedom to add complimentary processes that support other activities and roles that may become necessary or require assistance.

For example, scrum (a project management centric process) is compatible with XP. For teams that have greater project management needs (or an available project manager), scrum can be added to the development process. But if it's not needed, then there is no need to be encumbered by the extra process.

I think this encourages a "on-demand, just in time" addition of process. The advantage of this approach is that the development process is kept lean. Starting with RUP for is like managing your development process in waterfall fashion; you preemptively specify all your project's process requirements and devise a design.

Alternatively, active process refactoring, facilitated by techniques such as retrospectives, allow the development process to evolve with the needs of the team and their context. This allows a team to adhere to the YAGNI principle (You Ain't Gonna Need It) of simple and lightweight process.

Unfortunately, process refactoring does not have the benefit of automated unit test / change detection suites. In XP, such tools make refactoring safe; you get immediate feedback as to what you broke and what dependencies exist; process refactoring can be riskier.

For me, this idea feels right; it comforts and helps alleviate many of the concerns I have regarding applied agile development. I think the next step is to identify process "smells" and their possible refactorings. (I have done some work related to this. See my Implementing Agile Process wiki section or the Agile Process Pattern Catalog.)

Posted by csepulv at 07:49 PM


September 12, 2003
What qualifies as an agile process?
I have been struggling with this question for some time. I don't think I have an answer, but I do have an idea. An agile process is a process that promotes:
discovery
Requirements, risks, architectures, strengths, weaknesses, proficiency, and throughput are examples of project and team elements that emerge or are discovered in an agile process.
unencumberment
Either get out of the way of X or keep Y out of your way. Remove anything that hinders your workflow and the progress and success of the project.
feedback
An agile process provides constant feedback along many dimensions of the project. This may take the form of an XP team's velocity, dependencies during refactoring from unit tests, Scrum Burndown Chart, or the frequent evaluation of software by the customer from frequent iterations.
sustainable, consistent pace
The month sprint of scrum, 40-hour XP workweek are examples.
synergy
The practices support each other. For example, collective code ownership and pair programming support each other in XP.

Besides the "by definition" consideration, existing agile processes, such as XP and Scrum, fit the above guidelines. I have not used Mary Poppendieck's Agile Customer Toolkit or Lean Software Development Toolkit, nor have I been part of a Feature Driven Development project, though I think they also conform to my guidelines.

Furthermore, I think the agility of an agile process is a measure of how the process contributes to the efficiency and productivity of the project and team. This contribution doesn't imply success, but should promote speed and responsiveness.

Unfortunately, I think it is very hard to measure agility; it is inherently qualitative and context biased . For example, I have always felt that Feature Driven Development isn't as agile as XP. I think the reason is one of encumberment. The documentation and project mangement aspects of FDD seem to encumber the project. However, you can make the argument that FDD may better promote the process of discovery and management, from the customer's perspective. XP provides little guidance for the customer as to the discovery of stories that provide the greatest business value.

I feel these guidelines offer a different perspective than the elements of the manifesto. For example, communication and collaboration are desirable because they promote discovery and provide feedback. As I consider the experiences I would characterize as "agile", I am better able to articulate their "agility" in terms of these guidelines.

Armed with this idea, I will now reconsider my thoughts on such questions as "what is Agile Testing?"

Posted by csepulv at 06:31 PM


September 05, 2003
Software Architects and XP

I have been spending a lot of time thinking about testers and XP lately. It has caused me to start thinking about other typical roles and their place in XP. So, what about software architects and XP?

The short answer is that they have no place. I say this with some confidence, as I used to consider myself a software architect. (Over the last few years I have become less and less comfortable with associating myself with the moniker.)

The incompatibility comes from the basic assumption software architecture makes: you can successfully design an architecture for a system in upfront fashion. Besides being in direct contradiction to the practices and principles of XP, it is practically flawed. Software needs to change. In my experience, most people are very hesitant to change that which they have made large upfront investments in. I've seen (and at times unfortunately architected) systems where the architecture became an impediment; it had to be overcome and accommodated as the system evolved.

This doesn't mean software architects should be fired from an XP project. They will, however, have to change their perspective and work habits. On an XP team, they will be "just" another developer. Their architecture expertise will be utilized as they pair with other developers.

In my opinion this is actually liberating for the whole team. First, there isn't the elitist notion of hierarchal roles on the team. Furthermore, as with a technical lead, there is an assumption and pressure that the architect knows all the answers. In XP, this myth doesn't have to be maintained; the architect can now offer their advice and expertise when appropriate and defer to that of others when appropriate. In XP the architecture evolves and is owned by the team, not one person or group.

I can sympathize with the allure of software architecture. It can be very satisfying to create an elegant design solution for software. But it is also frustrating and stressful to watch the once-pristine architecture get band-aided and fail as needs change. Ultimately, I am a pragmatist. I would much rather construct working software than pretty diagrams.

Posted by csepulv at 06:53 PM


August 30, 2003
Comments in Code 2: Another consideration

Brian Marick, in a related blog entry, discusses an interesting perspective on the issue of comments in source code. As he frequently does, Brian brings another dimension and perspective to the topic. Brian discusses the expectations of your audience and provides an example of code in C, that if directly translated to Lisp, is not too easy to understand. The "mis"understanding exists because of the generqally accepted style for Lisp code, which is different from C code.

This caused me to think of another example. Lately, I work with C#, but still work with Java from time to time. Java has anonymous inner classes, C# does not. In many cases, anonymous inner classes improves the clarity and simplicity of the code. Given the similarities between C# and Java, I tend to read Java code with my C# hat on, which always causes me to pause when I read code that uses anonymous inner classes.

Should this code have comments, explaining the intention and use of the anonymous inner class? From one reader's point of view, the code may be clean and not need comments, but another reader, as in my example, might benefit from them.

However, I think it is a death spiral if you attempt to accomodate too many perspectives. Do you comment Java for C# readers? What about Java for Ruby, Lisp, Prolog, Python....readers?

Let's assume you only consider the experienced Java reader. On the refactoring group, I see many different opinions regarding the best style and constructs for any particular code snippet.

I don't think this justifies source code comments. Comments assume that the language (English for example) and the prose is readable by all, or at least more than the code. (There is also the issue of keeping the comments in synch with the code as it changes, which is one of my primary practical reasons against comments.)

These considerations cause me to realize something: both the definition of clean code and explanatory comments make assumptions about the reader, her context and her experience. This is true about any writing, but it is particularly complicated for the software developer. The expected future reader can vary dramatically but will probably be faced with some practical reason to quickly understand and use (modify or integrate) the source code and accompanying comments.

This leaves me with more questions than any conclusions. Though I still think a developer should challenge himself to improve his code before he writes a comment, I wonder for whom should the code be made more readable?

Posted by csepulv at 08:36 PM


August 28, 2003
Implementing Agile Practices

I have been ranting for a while (see this post) that the agile development literature is severely lacking. I am mostly concerned with the actual implementation of an agile process and all the real world issues that arise.

"How do you train a customer to be a good XP customer?" , "How can you be a good coach?", or "How do testers fit into XP?" are examples of questions that frequently arise when a team embarks on agile development. Most of these questions are very context specific, so I think it is premature to make any proclamations about them. However, there is a variety of relevant experience that could be useful to those considering such questions.

So, rather than continuing to whine, I am trying to do my part for a solution. I have set up a wiki at http://www.christiansepulveda.com/cgi-bin/moinwiki/moin.cgi/ImplementingAgileProcesses

where I have started to catalog my thoughts and experiences regarding these issues.

I am also mining the various Yahoo groups (and other sources) on agile development and adding links from the wiki to relevant discussions.

I hope the wiki will be a resource for those implementing an agile process. Who knows, maybe it will grow into an interesting article or, more ambitiously, a book.

I encourage you to browse the wiki. Email me if there is something you think is missing or feel free to add it. (Please just follow the wiki guidelines on the front page.)

P.S. There is a related project, at http://www.berczuk.com/wiki/bin/view/Agile/AgilePatterns. It is a catalog of agile process patterns. It is a topic I have been interested for some time. Unfortunately, we haven't made a lot of progress (at the time of this post), but there should be lots of entries soon.

Posted by csepulv at 12:15 PM


August 26, 2003
Comments in Code

I was reading a rather long winded comment in some source code and I started to hear the voices in my head rant about comments in code. This particular comment was written by a rather bright individual who insists on commenting his code. (To make matters worse, he uses C# regions to "hide" particularly long comments, reducing the comments value.)

Not all comments are bad. But they are generally deodorant; they cover up mistakes in the code. Each time a comment is written to explain what the code is doing, the code should be re-written to be more clean and self explanatory. If you are unsure how to make it more clear, get help .(Hopefully you were already pairing. If not, this is an excellent execuse to get another pair of eyes.)

Comments are appropriate many times. For example, comments can be used to note areas of code that need refactoring, but for practical reasons are being left as is. They may warn against changes, as a section of code may have been tweaked for performance reaons and shouldn't be changed. (Or at least the decision to change it shouldn't be made lightly.) They can note design decisions and tradeoffs made.

For example, I recently had a situation where 6 different possible actions can be invoked depending on the state of the object. This screams polymorphism to me, but I decided it would be overkill, in this case. The specifics are not relevant, but I commented the source code to indicate this decision. If someone decides to refactor it later, so be it.

So, before you use a comment for explaining complicated code, please think about a way to improve the code.

Posted by csepulv at 12:10 PM


Simplicty is Hard
Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius--and a lot of courage--to move in the opposite direction. --Albert Einstein

I have always believed in Einsten's quote. There is a related one:

Everything should be as simple as possible but not simpler.-Alber Einstein

The two quotes embody a basic software design philosophy that, when applied, results in good things: simple code is clear, maintainable, reusable, and extendable. But as Einstein notes, simplicity is hard.

This is is why Test Driven Development (TDD) is beautiful. When practiced with discipline, automated tests guide the programmer to simplicty; they provide a roadmap to nirvana. (Alright, this is a bit too flowery. I am starting to sound like an XP evangelist)

Achieving simplicty is still hard. But for software, TDD makes it a little easier.

Posted by csepulv at 11:56 AM


Testers and XP: Maybe we are asking the wrong question

Upon recent refelections of Agile Fusion and the corresponding forum discussions, I've had a couple of "aha" moments.

We have been asking the question, "What is the role of testers in XP?". Perhaps this is the wrong question. It is a little too general and implies its answer is prescriptive and would be a proclamation.

A better question might be "how can a tester contribute to an XP project?". A slight difference in wording has significant difference in implication. It is open ended; it is not prescriptive but exploratory. It is the type of thing Lisa Crispin addresses in her book.

Still, I think there is more to this. One element of XP that is appealing is its simplicity. Furthermore, I am amazed as to how many contexts can be "reduced" (one of the "aha" moments, see this post) so that XP is appropriate. But, XP doesn't address all areas of software development; XP is developer and code centric.

Metaphor, Simple Design, Automated Testing, Refactoring, Pair Programming, Collective Ownership, Continuous Integration and Coding Standards are directly about coding. The remaining practices (Planning Game, Small Releases, Onsite Customer and 40-hour Week) are about either staying out of the programmers way so they can code or improving communication about what they are going to code.

Anything that is outside the realm of "coding" is not really addressed by XP and I don't think we should try to change XP to address these other things. I think it would hurt more than help; it would complicate XP and is beyond XP's intention.

But, there are other agile practices that address these other concerns and work in harmony with XP. Scrum is the best example. Scrum is about project management, not coding. When I am asked about the role of project managers in XP, I suggest scrum. Scrum is another process that collaborates with XP; it doesn't really change or extend XP. I think this is a key reason developers don't have allergic reactions to scrum. It doesn't get in our way and we can pretty much ignore it.

But scrum is great for the customer and project manager; it helps deal with many of their concerns. So, I think we should consider what would be an agile process for testing, or rather figure out what is "agile testing".

I cringe a bit even as I type "agile testing", as there has been some disagreement already over its definition. Perhaps a different name will be better, but I am specifically looking for a process, framework or practice that adheres to the following guidlines:

  • is consistent with the agile manifesto
  • will complement, not impair, XP

I think there is enough experience among people like Brian Marick, Brett Pettichord, Lisa Crispin, James Bach, Andy Tinkham, Cem Kaner, Elisabeth Hendrickson, and others (sorry if I left anyone out), that if you were to mine your experience, you probably could come up with some ideas and patterns that would address the above. I think a lot of Brian's posts to his blog, are exploring this.

Part of the confusion for me has been the scope of testing. One dimension of testing is directly related to coding, and these facets can benefit from tester/programmer pairing. But so much of testing is not about coding, but exploring risk, for example. These "para-coding" activities and goals are outside the scope of XP, but could complement XP and therefore make for a more productive team and a successful project.

Posted by csepulv at 11:13 AM


"XP will never work at my job": Maybe it can?

I frequently read and hear about claims that "XP won't work for us" or "I can't see how we could do XP". XP isn't appropriate for every development context, but it works in suprisingly more settings than it may first seem.

There is a mathematicl technique known as reductions that I think provides a nice metaphor for this problem. Reductions are a theorem proving technique, in which you convert an existing problem to another problem. The other problem, which is hopefully simpler, is now the one you provide the proof for and therefore show that the given solution can be extended to the original problem.

There are many contexts where people assume you can't use XP. But if you try to find a reduction for the context, you frequently can filter out a lot of noise and complexity, such that you can see how the context can be simplified and XP can be used.

If nothing else, the process of thinking about your context from a different perspective will probably expose assumptions and areas for improvement.

Posted by csepulv at 11:03 AM


August 14, 2003
Naked CRC

At Agile Fusion, Michael Feathers demonstrated a technique he learned from Ron Jefferies, that Ron called Ironman CRC. While at Agile Fusion, a few of us used the term Naked CRC, which I think is much better as it is more memorable and better describes the technique.

This is really hard to describe unfortunately. It is much easier to demonstrate. The idea is that you layout index cards, without anything written on them, as you describe basic collaborations of the system. The cards serve as a visual guide to your discussion. The key idea is that you only focus on a few classes / objects in the collaboration. Besides making the discussion easier to follow, using only a few classes allows people to follow the significance of the cards, as they have nothing written on them. I like this because it forces me to keep things simple when I use Naked CRC. If someone tells me "Wait, what kind of object is this card?", I realize that I have probably over complicated the example.

I have modified the technique to use multi-colored index cards. I bought a pack of index cards that came in four colors. (Incidentally, in college I learned about four color maps and how four colors is actually sufficient to draw any map without having any two adjacent entities be the same color. So, four colors works well with Naked CRC). I find the colors are easier for people to keep track of the model being shown, without resorting to writing on the cards and making things too complicated.

Posted by csepulv at 04:22 PM


July 23, 2003
NBuilder: Iteration 0: Retrospective

Iteration 0 was really short, so there isn't too much to say. I had set an 8 hr time box which I cut to 3, since I was able to make a few decisions quickly and my research didn't take too long.

First, for the small logistics, NUnit and NAnt are definitely going to be used. I am going to pass on Cruise Control.NET for now. I was researched MS Visual Studio / CVS IDE integration and found this article, which gave pretty good instructions. After overcoming the frustration of not getting it to work in the past (and as the article's author points out), I realized the Microsoft SCC (Source Control API) is not really compatible with CVS. For example, you must explicitly check out files before working on them. I find CVS tools and semantics to be better, so I will stick with WinCVS. IDE integration doesn't get me too much, though I admit I really like using it in Eclipse.

The most significant part of Iteration 0 was selecting a project. I have decided to use the construction of NBuilder itself. I realize this is a little on the metaphysical side and a little "trendy" (using a tool to build a tool, like xUnit), but I was looking for a project that would allow me to explore my requirements and was realistic. I don't want to invent some fictitious situation, as it would be harder to consider the real needs of an application. I fear I would start to meld the app's needs to satisfy the assumptions I have about how NBuilder should work. Since the usage of the application should allow requirements to emerge, I think this is a good strategy.

More later, with Iteration 1 planning game, though I will be out of the country and won't touch this for a week or two.

Posted by csepulv at 06:28 AM


July 16, 2003
NBuilder: Iteration 0: Planning Game

QUESTION: I plan (hope) to make frequent postings about NBuilder to organize the project and provide a basis for reflection. For xTeamWorks I started a separate blog in anticipation of it becoming a distraction for anyone reading my normal blog. Anticpation is bad (not to mention my arrogant presumption that people read me blog), so I am not doing this for NBuilder, at least not yet. If anyone cares about this, let me know if you would prefer a separate blog for NBuilder or for me to post NBuilder entries on my normal blog. Thanks.

I have the following stories for Iteration 0.

Determine sample project
As NBuilder is a tool for persisting data in applications, I need a sample project.
Setup environment
Setup Source Forge account, build environment, configure change management, etc.

I plan to have this iteration be short (one day). Since I rarely have a whole day to spend on personal projects, I am time-boxing the iteration for 8 hours. In a retrospective, I will monitor my velocity for actual time spent working vs. the time that transpires due to lag between work sessions.

As far as the development environment goes, there are specific tasks / considerations. I will be using MS Visual Studio 2003, NUnit and possible NAnt. I have not used Cruise Control.NET and may look into it. I need to decide about configuration management. To this point, when working on open source projects with .NET, I have used Tortoise and WinCVS to update files. I want to use something that integrates with the IDE, but have not had success setting up Igloo yet to work with SSH. I also have the complication that a lot of my work is done offline, be it on planes or in hotels. I like to use MS Visual Source Safe locally to provide version control when offline. Ideally, I would have source control IDE integration, that I can use with CVS when online and switch to Source Safe when offline. (It would be really nice to synch the incremental checkins from Source Safe to CVS. If anyone has any ideas or experience with this, please let me know. I will be very grateful.

Posted by csepulv at 07:00 PM


Vision for NBuilder

In a previous entry, I noted that I am beginning to work on NBuilder again. As I am starting a new, I am trying to be quite disciplined in adhering to the principles of XP and TDD. (I hope to be critical and consider if I follow these principles as well as I presume to.) So, in order to provide a monitoring and reflection mechanism for myself, I will try to diligently blog my thoughts.

These are my primary goals:

support testing
I want to use ideas like ObjectMother so that I can have data driven / dependent tests run fast (under 1 minute rather than more than 20)
persist objects easily
This one is hard. There is tension between making constraints on objects, aggregation methods, and attributes and simplicity of the framework. On one hand, I want the objects (and their needs) drive the model and then deal with their persistence. On the other, I want to reuse tools rather than re-craft and I want to eliminate the tedious code that can accompany implementing persistence support.
support extensions
I'd rather have a small basis to reuse and tailor on a case by case basis than a bloated tool that doesn't fit any actual context well.

There is one significant requirement / motivator for NBuilder, that I don't want to make a core goal but is in the back of my mind. I would like the persist the objects with different mechanisms, such as XML vs. a relational database. One of the practical drivers have been the need to refactor and tune database schemas, be it for performance or space consideration.

There are a variety of other specific features in mind, but I don't want to clutter my thinking, get ambitious or do much anticipating. I want the vision to be simple.

Posted by csepulv at 07:00 PM


NBuilder: It Begins..(again)

"NBuilder is an application framework for developing enterprise and information system applications. It provides support for the development of an object-oriented persisted domain model. It includes tools for modeling the domain, generating code for the domain entities and application, and a series of base classes that allow the rapid development of a stable, scalable, high performance application that easily supports automated testing and reusability."

The above appears on the Source Forge site for NBuilder and pretty much reflects my original ambition. Basically, most of the code I write needs to be persisted in a database and .NET doesn't have the nifty tools for object-relational mapping that Java does. (I can't believe I used the word nifty!) So, I (and the teams I have worked with) have generally crafted custom code generators / metadata managers that provide persistence and related services for Microsoft .NET projects.

There are a few motivators for me. First, I hate duplication. I (and other team members) have built such a custom tool several times now. Each time we learn something new, experiment and tweak for the specific circumstances in which we were working. So frustration is the first motivator.

Impatience (and frustration again) is next, as I can't wait anymore for someone smarter than me to do this. I have come across a few tools that address similar issues (some free and some commercial), but they don't address my needs terribly well.

Finally, there is vanity. Part of me (a larger part than I really want to admit) is convinced I can achieve some level of generality for the tools application. I actually tried this a few months ago, wrote something fairly quickly that I thought was useful. (It is actually being used for production code by one of my clients.) It has a variety of flaws, which I won't go into here, except for one. The largest flaw was in the approach and ambition. I assumed to know too much (I have done this multiple times before, I can fast forward to..) which leed to anticipation and some flawed design decisions. I also started with the ambition of writing too general a tool.

So why will this time be any different? It probably won't. The rationale part of me can smell some bad signs already. But, there is the vanity thing and vanity isn't rationale. So, being a glutton for punishment, I being again the search for my white whale.

There are a few things that keep me hopeful (or delusional). I think of Kent Beck's reflections on the money example of Test Driven Development. Beck notes how he has written the example numerous times and found that with a new metaphor, he discovered the cleanest implementation yet. I am hoping the powers that be take pity on me and provide this inspiration. I also hope to be as disciplined as possible in XP and TDD principles as I can (which I did not do so well last time around). Next is a practical hope. I have a few current "issues" I need to address in a couple of different projects. Some are client related, some are not. This will limit the amount of disclosure I can provide in my blog entries, but being a pragmatist I will quickly abandon all lofty ambitions for solving the problems that pay the bills. If I do this well, this attempt will definitely make it worthwhile.

Finally, I will learn something, probably a lot of somethings. This new knowledge may prolong my vanity and delusions, but continued learning is better than being stale and complacent.

Posted by csepulv at 07:00 PM


July 10, 2003
Emotional Resistance to Agile Development

During a recent dinner conversation in my home, one of my guests asked what I do and I started talking about agile software development. There was a general response as to why doesn't everyone develop software this way, as if it was so logical it would be common sense.

The reaction startled me a bit, especially since I see so much resistance to agile software development. I think much of this resistance comes from the emotional baggage of those considering agile development.

Many people considering agile development are doing so because they?ve had bad experiences and problems with software development. This problematic history is marked by dysfunctional work environments, failed projects, missed deadlines, extreme pressure, blame assignment, and general tension.

For example, a common management opposition to pair programming is "Why would I pay two people to do the work of one?" This frequently can be interpreted as "I already pay two people to fail to do the work of two people, now you want me to pay two people to fail to do the work of one person." There is so much distrust of the developers? productivity that management may fear pair programming granting permission to produce less.

Another example of emotional resistance is from a project manager. He may wonder, "Do I still have a job in XP?" or "How will people know I am doing a good job?" Developers have similar fears. In a pair programming environment, there is less room to be the "hero" or show off programming prowess. You can't take individual credit for a successful application when there is no code ownership.

Sometimes the resistance comes from the transference of previous experience. For example, I once outlined an automated testing approach to a group and someone reacted very badly. He flatly claimed it would not work and had no value. It was revealed later, he had tried something similar and it failed. He identified the two ideas as equivalent and had a "it's happening again" reaction, which blocked his ability to assess the actual approach outlined and not just his assumption of it. Furthermore, if the approach I outlined did work (and it really was similar to his own), then he is faced with confronting a new type of failure; maybe he was the flaw and not the idea.

In all these examples, there is an emotional resistance to agile development. The irrationality of emotions requires a different approach; you can't simply present a logical framework and expect that reason alone will be sufficient to generate acceptance.

I often hear "How can I sell agile development to...?" Maybe it is better to ask, "Why aren't they buying into agile development?" The human component of a team is much more important than the process. The better one understands the human dynamics of a team the more successful the project will be.

Posted by csepulv at 10:59 AM


July 09, 2003
xTeamWorks: In the beginning...

Bill and I started to work on xTeamWorks. I am not sure what is the best method of organizing all of this and I don't know how many people will actually be interested in reading all of this. Anyway, this will be few of several cross posts between two blogs, my normal blog and the xTeamWorks' blog.

There are several sources of information:

Source Forge Site
This site will contain the actual release files, product documentation and all product release information.
xTeamWorks' Blog
This is a journal of the project, how things are going, my ramblings, reflections, etc.
xTeamWorks' Wiki
This is used to actually manage the project. We will keep track of our XP stories, iterations, velocity and tasks.

The reason for all the different sites is that there are three different views of this project: the final product site, the management of the project and my personal accounts of the experience. This project is intended to serve multiple purposes:

work with XP in a remote setting
Bill and I don't use pure XP lately and I tend to work remotely on my current project. Since I must presented a paper at ADC 2003 about remote agile development, I wanted to explore a pure XP / pure virtual project. Though this is small, I think open source projects can benefit from remote XP.
build a tool to support online XP management
I work remotely a lot so posting 3x5 cards on a wall doesn't work for me. I did look at some other products for managing XP online, but they didn't quite address my needs or I couldn't get them to install correctly. So, like any good developer, I have convinced myself I can do a better job.... ;)
work more with Java and Eclipse
Bill and I have been mostly in Microsoft / .NET land for a while. I haven't done much Java work over the last year and haven't used Eclipse. This is a chance for us both to explore.
Posted by csepulv at 12:09 AM


July 03, 2003
ANN: xTeamWorks: An XP Project Tool

I am starting work, with the help of William Rakocy, on a web based collaboration tool for managing an XP project. I work remotely quite often so it would help to have online access to story cards to other project items. I have experimented with some wikis, MS Excel files and other techniques, but I think I could I could benefit from a tool.

I experimented with XPWeb and XPlanner, but wasn't particularly satisfied with them, at least for the needs of my project.

So, xTeamWorks will support the basic elements of XP, like story maintenance, task definition and signup, velocity tracking and iteration planning.

xteamworks.sourceforge.net is the website for the project. I will probably post updates as we go here.

If anyone has thoughts or requests, let me know.

Posted by csepulv at 08:18 PM


Agile Development and Remote Teams: Learning to Love the Phone

I recently presented this paper at a talk at Agile Development Conference 2003.

Agile Development and Remote Teams: Learning to Love the Phone

I'd love to get people's impressions and feedback.

Posted by csepulv at 05:39 PM


ANN: New Yahoo Group on Remote Agile Development

Based on interest to my talk at the Agile Development Conference on remote agile development, I have started a yahoo group on the topic. I encourage you to join the group if you have any interest of experience with agile software development, remote teams or just want another group to read.

Go to http://groups.yahoo.com/group/remoteagile/

Posted by csepulv at 01:41 PM


Preemptive Optimization: It doesn't only apply to code

I had a discussion recently with someone about something I read in the Pragmatic Programmer, by Andy Hunt and Dave Thomas. They write of the importance of developing proficiency with keyboard commands and text editor, to increase productivity by reducing keystrokes and mouse movements. There is merit in the book's discussion of the topic and I don't wish to dwell on the argument they make.

My belief is that preemptive optimzation is bad. Martin Fowler has addressed this a variety of times with respect to code and design. The idea is that you shouldn't try to preemptively increase the run-time performance of code design, as the bottlenecks in an actual system frequently are not what they were pressumed to be. You should profile the system and then tune the code to the actual bottlenecks of the sytem.

I feel this applies to people and processes as well. The advice offered by the Pragmatic Programmer is more of a technique than a goal. It may be nice to show off lightning fast keyboard skills, but it doesn't necessarily improve individual productivity, and is less likely to improve team productivity. Given there are only so many hours in the day, I advise that a programmer (and team) should reflect on her practices and think about where she can improve. If keyboard and environment proficiency seems like a good idea, then go to it. But it may not be advisable and may be a waste of time.

Like I feel with most things, just make sure you don't take any advice as an escape from thinking and reflection.

Posted by csepulv at 01:36 PM


June 26, 2003
Integrating Testers into XP: Initial Thoughts

I was at Agile Fusion last week, an event intended to explore the ways testers could be integrated into XP, and though there was a lot of good dialogue, there was no resolution. I am not disappointed by this, as this issue is not something you simply "solve" in one week. But it has left me thinking as I believe skilled testers can greatly improve the agile development process.

During a tutorial I attended today about the Lean Development Toolkit, I found inspiration. Mary Poppendieck talked about concurrent engineering in the automotive industry and commented that software development could benefit by adopting the technique. In concurrent engineering, the product is developed with multiple, parallel activities aimed at developing different components of the finished product.

So, I have the following idea, though it is really half baked. Consider the beginning of an XP project (or any XP iteration for that matter). During the planning game, testers contribute by asking questions that help expose areas of risk and necessary consideration. Though the whole team should be doing this, skilled testers are particularly adept at this. (Lisa Crispin raised this idea during Agile Fusion.)

The rest of the planning game proceeds as normal. Once stories are established and tasks laid out, development begins with normal pairs of developers, with an occasional third member (a tester) who can move among pairs, sort of like a bee collecting pollen and cross pollinating. The tester, at this point is observing more than participating, but she is gathering information about the internals of the software, the habits and biases of the programmer, the architecture decisions, etc.

The tester only does this gathering for s short period of time, at which point the tester (or testers, though I have though about this much) break away from the development activities to focus on their activities. From what I understand of context driven testing, such testers will formulate theories about how to test specific systems, effectively developing strategies and plans for testing the system (though not huge documents of test plans and specifications, but rather ideas and strategies).

At the end of the iteration, the tester will discuss with the entire team his theories about how to test the system, the important risks and any concerns they have. This information is a vital part of the iterations retrospective and this feedback should be used for the next planning game. The developers hand off a working system to the testers and begin their work in the next iteration. The testers have a working system to test; both for their approaches of how to test the system and the system itself. They revise their plans, expose new risks and identify defects (or areas for improvement). The testers should still join some pairs to constantly learn about the system internals and the developers should integrate the contribution of the testers in their automated tests. This workflow continues for each iteration.

I would characterize this workflow as a weaving in and out of the tester with the pair programming activities. It allows the testers to collect information that helps them better formulate their test strategies, but gives both testers and developers the freedom to concentrate on the tasks that they are best suited to.

Brian Marick (and other in the context driven school), commented that one of the core principles of context driven testing is that testers would rather have working software to test than documents about how they would test the software. He also noted that context driven testing develops testing strategies appropriate to the system they are testing. I think that the workflow I outlined satisfies both these desires; testers get working software in each iteration that guides the refining of their strategies. The developers also get feedback, with each iteration, about what issues they need to think about and hopefully use this information to improve their automated tests and overall code.

There is another element that appeals to me in this idea. I think developers prefer to build something first, then reflect on it, and integrate their reflections into the software, proceeding in a very incremental and iterative fashion; developers are anxious to dive right in. Testers (from what I have observed), prefer to think about the system, consider it from many perspectives and generally develop a hypothesis before they actually start. I think both perspectives are appropriate for their task. A developer's job is to produce software, so active development directly supports the production of working software. A tester's job is to inspect software, rather than produce it. The tester wants to formulate a strategy first to avoid random "hacking" testing that is undisciplined and immature. There is conflict if you try to force the developer to spend too much energy and time before they get to write code and not allow the tester enough time and thought about how they would test a system.

So, I think the approach I am outlining allows both developer and testers to work together without getting in each other's way. The "weaving effect" is just enough collaboration to provide a benefit without trying to force one to emulate the other's job.

Another benefit is that it provides an iterative and flexible environment for testing. The testers are integral members of the process, and with each iteration they have a more functioning system that they can test and revise their approaches with. I would hope this increases quality and the efficiency of the software and team.

At this point I am starting to babble and it is getting late. I have lots more ideas in my head, such as pair debugging (something I have done with testers), but I want to get feedback. I have already discussed some of these ideas with others. There seems to be enough merit to continue the discussion, though I don't know how much.

Posted by csepulv at 11:28 PM


Lean Development Toolkit at ADC 2003

Lean Development Toolkit

I went to Mary Poppendieck's tutorial on the Lean Development Toolkit. The following is a sampling of ideas that were most important to me:

An organization's ability to embrace change and deliver fast is a competitive advantage. I see this as a core motivation of agile development.

Software is a development activity, not a production activity. Production activities don't benefit from iterative activities, for example, where as development does.

A resource that isn't 100% utilized is not waste. This is a common misconception. For example, consider a piece of machinery. It may operate optimally at 80%; anything higher can cause burn out. You wouldn't want 100% server utilization. So, don't try to use human resources 100%.

Cost of Delay: On Time Commercialization: Explores a P&L statement that assesses the cost of delay in software. The model is an estimate and intended to serve as a guide for making trade-offs. It is formed with the help of a cost accountant and not intended to be an accurate.

The GE Workout: Don't tell people what to do, ask them. Change the culture: speed, simplicity and self confidence

You cannot repeatedly deliver quality, in a timely manner, without discipline. (I include this because during an open space discussion last night someone tried to argue you don't need discipline to do XP well)

Posted by csepulv at 01:11 PM


It's Okay not to Run so Fast

I don't think all development teams should always try to go faster. I think they should always try to improve, but this doesn't always imply speed. I am responding to an attitude I see prevalent in the agile, particularly XP, community, where speed is worshipped. It isn't that I think speed is bad, but too much of an emphasis on speed can limit your thinking. It may be easier to explain my concern with an analogy.

I frequently compare fitness and dieting to agile development, particularly transitions to agile development. If you have been lethargic for a long time, your goal may be to look like someone on a magazine cover, but this won't happen overnight. More importantly, you shouldn't try. Extreme dieting and massive exercise will place too much stress on your body. You shouldn't try to go too fast too soon.

In both the fitness and agile development scenarios, I think it is important to outline your expectations. Perhaps the development time cycle isn't as much a problem as the defect rate. Focus on techniques that improve code quality rather than speed to meet those expectations. Even if speed is a goal, your team's velocity is probably not going to improve ten fold after iteration 0. I think too many fledgling adopters of agile development underestimate the struggles they will encounter. (I think the basis for this, as with fitness, is that most people don't want to admit how bad shape they are in.) An experienced coach, for example, will address this, but I don't think the literature about agile processes addresses this well.

So far, I have only noted why it is okay to not to focus on, or expect, speed in early stages. What about mature agile teams? Returning to the fitness metaphor, many people discover that they are comfortable not looking like the magazine cover model. They realize the cost doesn't justify the benefit; they like eating Twinkies. Similarly, the agile team may have certain cultural, contextual and organizational factors that limit their speed. In certain circumstances, it is more important to maintain these constraints and accept the limits they impose so long as the organization adjusts their expectations to accommodate these limitations. Acceptance and contentment are necessary for happiness.

Mary Poppendieck talks about delivery speed and competitive advantage. The core idea is response time; this doesn't imply that increased programming velocity will result in better response time. It ignores all the other factors that go into shipping a software product. These factors, since ignored, are at a higher risk for being the bottleneck than programming speed.

I have seen many martial artists, who may not look fit and can't run very fast, but whose reaction times seem inhuman. They are flexible and responsive; they are agile in the truest sense. I think agile teams and organizations should learn from this example and look for all the ways that response time can be shortened.

I don't think speed is a bad thing. Its just important to think about how you define speed and what motivations and pressures drive your search for speed.

Posted by csepulv at 10:24 AM


What's missing from Agile Processes?

Yesterday, during an open space discussion about introducing agile development to an organization, questions were raised about what is the role of product managers and project managers in an agile process. Most agreed that this isn't addressed in the literature (at least to the group's knowledge of the higher profile writings) and it is one of the examples of why you need experienced help when beginning the transition to agile development. There are a lot of activities and roles present in real, working situations that are ignored in the literature.

Project managers are addressed somewhat in discussions of Scrum, for example. But I have read very little about the role of the product manager. Though many agile processes shy away from over specifying of roles (which I agree with), there is little guidance about the supporting activites these roles play. My concern, is that a naive or literal adoption of the processes outlined in the literature drops a lot of necessary work on the floor.

For example, a product manager typically collaborates with marketing to discuss schedules, the creation of marketing collateral, and launch plans. Many may argue these are outside the scope of development activities, that they are business activities, and not central to an agile process. What about product documentation? It isn't written by developers, but requires intimate knowledge of the product being developed. What about testing? There is a lot written about automated unit testsing and a little less about automated customer acceptance tests, but this is only part of the testing landscape, as I learned at Agile Fusion last week.

One idea I have that is half baked (as is this entry) is that if agile processes tried to address all the elements that necessary to ship a commercial product, for example, the result would resemble the Rational Unified Process. This would bring us back to where we are trying to get away from. I think the real value of all agile processes is a pragmatism that encourages flexibility and adjustments, rather than prescriptions and ceremony. Ken Schwaber, when talking about Scrum, highlights the feedback that is provided by Scrum (and other agile processes) to guide you to your destination. In an agile process, you know the goals (destination) but you don't know the road you will take to get there at the start. This is better than mapping out the entire journey in advance only to find a bridge along the way has been washed out.

So, I am left with the following desire or challenge. I would like to see some of these issues discussed or addressed, in the same fashion as processes such as programming has been addressed. I want guidelines, tools, mechanisms, heuristics, frameworks, techniques or advice related to issues like testing, product docunmentation, marketing, and all the other elements of software development that aren't directly related to programming, but necessary to ship product. In all the conferences and events I go to, these issues are frequently raised by those considering agile development. It seems like the literature ignrores or hides from these issues. When confronted, an agile advocate typically responds with some comment about "Well in real practice..." or "Oh yeah, you do..." which addresses the practical necessity of these issues. So, why is such a pragmatic philosophy hiding from such practical issues?

Posted by csepulv at 09:09 AM


From the Inaugural Agile Software Development Conference

I am at the first Agile Software Development Conference this week. I am excited because I am really interested in agile development and I will be presenting a paper on remote agile development, a topic of particular interest to me (more on this in a future posting).

I am hoping to record and reflect as I attend the conference, so I apologize if these postings are a bit rough.

Agile Contracts

I started the morning with Agile Contracts, a workshop held by Mary Poppendieck and Christine Moore.

Keynote

Jerry Weinberg gave the keynote. He talked about what it takes to be a good professional.

Open Space

I went to two sessions:
  • How to be champion agile processes for the non technician
  • Can agile remote development work for legacy code?

Agile Contracts

I started the morning with Agile Contracts, a workshop held by Mary Poppendieck and Christine Moore. It was a good workshop and the following is a summary of what

Different goals for the Contract

There are basically two goals for a software:
  • protect against possible opportunistic behavior of the other party
  • promote mutual benefit from the project

This was best demonstrated by a prisoner dilemma simulation. (For those not familiar with the prisoner dilemma, it is a sample problem in game theory. Search the net for more info.) In our exercise, some groups defected, while others cooperated, then a repeat game resulted in all defections. The take away lesson is that the defections represent a risk mitigation strategy to protect you from the opportunistic behavior of the other party. A contract based on this principle can signify a bad start to a relationship, as both sides are anticipating the need for protection mechanisms against the other right from the onset; it stages an adversarial relationship. (A comment made several times was that if you find your team referring back to the contract, it is smell of dysfunction and potential problems.)

The alternative is to formulate a contract who mechanisms seek to promote mutual benefit. The complication, as I see made evident by the prisoner's dilemma, is that people tend to gravitate to the risk coverage strategy and anticipate lowest cost rather than maximum gain.

We then discussed various types of contracts, mechanisms for negotiating and related experiences. For more info please see the website.

Keynote

Jerry Weinberg gave the keynote. He talked about what it takes to be a good professional. He remarked that most people look at the "self" perspective, but ignore the "context" perspective and "others" perspective.

Context

Observation
Jerry remarked how it is hard to for managers to observe how people are being effective in an agile process and this can make them uncomfortable. He noted an anecdote where a manager expressed that in an agile environment, with elements like pair programming, the lack of code ownership makes it hard to know who to blame. Jerry's response was a seemingly rhetorical question: Which would you rather have, a process that works but you don't know who to give the credit to or one that doesn't work but you know who to blame? The manager responded that he would prefer the one that doesn't work, but he knew who to blame. (The manager, who had so many failed software experiences, expected more failure and therefore wanted accountability.)
Service
We generally don't get paid to develop software for ourselves, but rather software that someone else will use. We are providing a service.

Self

Work Habits
Reflect on your work habits.
Learning
Try to make sure you learn something new everyday. Always try to improve.
Self Care
Make sure to get rest and play.
Attitude
Are your fearless? Committed? What is your attitude toward your work?
There was one other item but I didn't make a note of it and can't remember.

Others

Communication
Obviously (hopefully), real important.
Credit
Share the credit.

Open Space

How to be champion agile processes for the non technician

There was a lot of ideas, but the discussion seemed to center on a couple of key points:

  • it helps (is necessary) to have experienced members (coaches, consultants) in a transition to agile development.

Can agile remote development work for legacy code?

There was a lot of experience stories and discussion. A few of the basic ideas were:

  • pure virtual development, where there has never been face to face interaction at some point, doesn't work
  • multiple forms of remote development
    • core is co-located, few remote developers
    • multiple, distinct distributed teams that each are co-located
    • complete distributed development where all members are remote

The convener of the discussion described his current context, which has four separate offices (the result of mergers and acquisitions), with some coupling and dependencies between offices, lots of legacy code and distributed management: project management in one place, senior technical executives in another.

My opinion is that he has three separate problems, with possibly related solutions:

  • how do we introduce agile development into this organization
  • how do we deal with legacy code
  • how do handle remote development

My opinion is that the real problems lie with the organization, whether or not its ready for agile development and the legacy code. These issues would be a problem if all the teams were located in the same building. This is part of a bigger idea that I tend to rant on: that a remote configuration can be implemented rather easily; the real problems are independent of the remote configuration and are related to the normal dysfunctions groups face. I will talk more about this in a later posting, as it is part of my experience report on Saturday that I am presenting.

Posted by csepulv at 07:31 AM