This is a long overdue announcement as I joined Pivotal Labs in 2006, after I left Nominum. But we are trying to raise our profile recently as we no longer want to be “the best kept secret in software.” (This is a quote from more than one client.)
Anyway, check out the following links. (In particular, note my personal Pivotal Labs blog as I plan to blog mostly there.)
April 9th, 2007
Most software practitioners I’ve met think the choice of technologies (programming languages, tools, libraries, etc.) has the largest influence on a project’s success. I actually think it is a minor influence and I think the project’s alignment with market demands is the single largest determinant of success.
While there are those who would explicitly claim the significance of technology, when asked directly developers may cite another factor. However, I make my claim regarding this “conventional wisdom” based on behavior; in most any project I’ve been involved in, a tremendous amount of time and energy is dedicated to selection of programming languages, technical evaluations, and debates about architecture and patterns.
By contrast, it seems the topic that gets the least attention is a very simple question: Are we solving the right problem? It amazes me how often, because of lack of interest or a strong sense of faith, developers do not question why they are building the application they spent countless hours working on.
From a separation of roles and responsibilities perspective, it seems appropriate that developers should not concern themselves with such a question; it is the job of the product manager. However, I’ve seen the most successful teams consist of a cross functional group, where each member took an active interest and responsibility for the project’s outcome. (Anyone been involved in a technical success, but it was a business failure?)
Agile software development can help a lot; the emphasis on communication, feedback and iterative development encourages the identification of real user need. It is amazing how much clarity can be achieved when developers and users sit together in the same room or you demo an application every two weeks. Alternatively, I find that traditional, specialized and compartmentalized development models, where information is “handed over the wall” or communicated with documents instead of conversation encourage a very myopic view by developers. Even if you wanted to better understand the business goals, pressures and motivations for the project, there may not been anyone accessible to ask.
Though I make a living being a proponent of agile development, giving too much significance to the process is just as misguided as emphasizing the technology. Agile development can be an effective means at understanding market needs, but it is not an end unto itself.
If I were to quantify my idea, in terms of percent influence on success, alignment with market needs accounts for 65%-75%, process, practices and team dynamics for about 20%-25% and technology for only 5%-10%. Similarly, I think the time and effort spent evaluating issues or devising tactics should be proportional to their importance. Simply put, I’d rather spend three weeks training a development team about user’s daily activities than three weeks evaluating the choice of Java vs. Ruby or SOAP vs. REST.
I’ve seen numerous examples where the development group had horrible practices or made many poor technical choices; it was amazing they ever got anything done. Yet, their products were great successes and wins for their organization. I’ve also seen cases where the architecture was beautiful and most any developer would awe at its magnificence, but it was flatly rejected by their user base. Unfortunately, I’ve even been involved in a case that was a marvel of agile adoption(well, at least the engineering practices): thousands of tests running in seconds, pair programming, and elaborate continuous builds: but it didn’t do much for the company’s bottom line.
Ultimately, you can’t win if you cross the wrong finish line.
April 20th, 2006
A good friend of mine, Iraklis Kourtidis, coined the phrase Costco-style programming. He described it by example: “I’ve had spinach in one of my dinner dishes each night this week. Not because I particuarly like spinach or it actually went with the meal, but because I bought a big bag at Costco and now I have to use it up.”
He was describing the software tean’s use of annotations, a C# feature that was new at the time. Developers frequently abuse some tool, trick or feature because it is easy and available.
Similarly, I described the new-toy effect as also motivating developers: you have a shiny, new toy available and you want to play with it everywhere and anywhere. Who cares that is doesn’t make sense that Godzilla would fight an omnious basketball; I’ve got a new Godzilla toy and he’s gonna fight everything.
The combination of Costco style programming and the new toy effect can be very dangerous for a software application. Developers would spam code bases, add useless, complicated features and even develop entire products motvated by these pressures. Though a completely different rant, I think much of the Internet Bubble of the late 1990’s can be explained by this; the Internet was new and available and lots of kids wanted to play with their new toy everywhere and anywhere. And the effect is worse when many went to Costco and have new toys.
November 20th, 2005
Every now and then someone makes a call for collective brain power; it may be someone in a blog posting or a CEO addressing his staff. The odd thing, at least to me, about such requests is that they frequently occur after some suboptimal individual attempt at solving the original problem. This seems inefficient; why wait as a last resort to call on the power of a collective conscious?
The simple answer is one of resource allocation and opportunity cost; it is inefficient to always point some collective at every issue when a subset can solve the problem. However, agile environments offer an alternative. The high visibility and communication of requirements, risks and decisions provides the opportunity for an opt-in application of collective brain power. Team members can choose to give background thinking to a topic, while not consuming everyone with it.
This isn’t necessarily a fully baked idea, but something I’ve observed with more than one agile team. It is not often there is the explicit call for ideas, yet is seems like many decisions have the benefit of input from multiple people. Besides being egalitarian and empowering it generally leads to good outcomes.
November 16th, 2005
Okay, so my title may be little exaggerated, especially since there isn’t officially an Agile 2.0. However, for the agile development community, there are many shifts, trends and new ideas that make it feel like a new generation of agile development. (More on this later.)
Anyway, I’ve been following some of the Web 2.0 developments lately; I’ve read many blog posts (http://web20workgroup.com is a good place to start for those interested), heard gossip about VC funding, and know a few people working on Web 2.0 projects. All of this information makes me think an agile approach is perfectly matched for the development of Web 2.0 applications.
Agile development emphasizes the need for feedback, discovery and adaptation. A common agile credo is “release early, release often.” Agile development is basically a genetic algorithm (for more on this, read this post); an agile approach evolves its software. Evaluating the “fitness” of the application with each iteration is critical. Real user feedback is one of the best indicators for making such an evaluation.
Most Web 2.0 projects (or at least those I have seen) are consumer oriented. Consumer oriented, service-based software markets are generally accommodating (and sometimes welcome) frequent releases, assuming each release is at least as good and as stable as the previous. Agile development favors short iterations and core practices such as automated testing and continuous integration develop the ability to release early, often and reliably as a core competency of its adopters.
Most projects in early stages are based on a vision and skeleton of a possible solution; success is largely determined by how quickly the application can evolve into what user’s need. Quick delivery is necessary to compete and not miss windows of opportunity. I think the best software process to support this is based on an agile model. One of my former clients (when I was a consultant) believed this as well and feels that their agile adoption was a key facilitator of their IPO last year. I also know some very large, high profile, Bay Area web companies who are adopting agile software development and see it as necessary to remain competitive.
I think others support the complementary nature of agile and Web 2.0. The canonical book on Ruby on Rails, one of the popular Web 2.0 enabling technologies, is Agile Web Development with Rails and it describes how Rails development embodies and accommodates an agile approach.
I noted that I believe the agile community is undergoing Agile 2.0. Last year the 2nd edition of the XP white book was published, in which Kent Beck revised and updated his description of XP. Mary Poppendieck’s keynote address at XP/AU 2004 described agile as “crossing Moore’s chasm”; it is shifting from early adopters to mainstream. While some feel this shift may water down agile, I think it can amplify its effectiveness; many in the agile community have updated and evolved their ideas to better support the software development process.
Finally, I have also adopted a more mature understanding of agile development. I’ve had many successes in my career applying agile methods, but have been challenged in my current job and our agile adoption. These challenges have caused me to reflect on many assumptions and ask many questions. Though I have more questions than resolutions (which I think is a good sign you are actually learning), my understanding and application of agile software development has evolved into its own “next generation.” I know a few others who share many of my ideas and are working in the Web 2.0 space. For these instances, I am excited to see how the combination of Agile and Web 2.0 turns out.
November 16th, 2005
Agile development is structured on iterative and adaptive development. The application evolves over time and, at least hopefully, closer matches the user’s needs. The basic process of agile development is a genetic algorithm; after each iteration, adaptations are made and the whole process repeats with each iteration.
The success of a genetic algorithm generally depends on the quality of its fitness function. A fitness function evaluates the success, or “fitness”, of a given result. Within agile development, the iteration boundary marks an opportunity to evaluate product backlogs, prioritize features, and generally tweak the process; both the product and the process evolve.
In order to have a successful agile project, it is imperative to have fitness functions for evaluating both the application being developed and the process by which it is developed. The agile community provides various guidelines for the process evaluation; retrospectives, cost of change, quality of team dynamics, test suite run time and size, shrinking code bases, etc. are all assessment tools and indicators of process health. However, there is little information about evaluating product health.
There are some obvious indicators and questions: do users like it, are bug counts low, is the product selling? I think these are symptoms of the relative health of the application, but I think you need more from a suitable fitness function, particularly when the product isn’t an instant and obvious success.
For example, prioritizing features can be a very difficult activity; users will typically assign many features a “number 1″ priority. However, ranking each feature in strict order, with no duplicate priority, is a challenge.
While the assessment of the product’s health can be seen as a marketing or product management activity, I do not think it is outside the scope of agile development. Successful projects depend on the collaboration of cross-functional teams. Thus, I think it is important to communicate the fitness criteria for an application amongst all on the team, even if one group has primary ownership of it.
Ultimately, the fitness function for an application is an answer to the question of “why do we build the products we do?” Do you know the answer to this question for your current project? Ask others and you may be surprised at the responses.
November 11th, 2005
Superficially, there are many ideas and practices that are in common with agile development and waterfall development. There is a Agile Unified Process, for example and an “XP plugin” for the Rational Unified Process. However, I think agile development is quite different than waterfall, though it is not always easy to name the differences and understand why drawing strong distinctions matters.
I think agile and waterfall are each based on a fundamental assumption; if you subscribe to this idea then you probably should adopt that process. The assumption is about how to achieve a successful software project.
For waterfall development, I think the basic assumption is that perfect planning leads to optimal results. Agile assumes is that an adaptive and evolving approach is the way to go.
I do not think anyone who adopts waterfall actually thinks perfection is attainable, but I think the entire process is aimed at the pursuit of a perfect plan. Extensive requirements, complete with test traceability, elaborate architectures, long analysis and design phases, and up-front test plans are attempts at precise, accurate plans. Usability, overcoming technical challenges and meeting deadlines are achieved (or so is the idea) by building a plan that satisfies these requirements.
By contrast, agile development tends to be lightweight on planning. There is a general pragmatism to “do as little as you can get away with” when considering planning, requirements capture, architecture and documentation, as emphasizing planning would emphasize a different base assumption. However, agile development, in particular XP, is very heavy weight when it comes to automated testing; tremendous effort is usually expended. The reason is the emphasis on adaptability and evolution. Agilists assume change is a necessary and inevitable and agile processes encourage developing your “accommodate-change muscles.” Thus, automated tests tend to be a primary mechanism to achieve efficient regression testing, which accommodates code design changes, for example.
Anyone who reads my blog knows I am very biased towards agile development. Though it may sound arrogant and obnoxious, I think many of the practices of waterfall are based on myths and thus make the process flawed. For example, the practice of software architecture is the application of learned patterns and principles in an attempt to satisfy functional and non-functional requirements. The problem is that technology changes quite fast and the software industry is too immature; it is hard to successful apply experience when the rules, constraints, and contexts are in flux. For at least the foreseeable future, I think you are better served with a process that allows for discovery and adaptation than depends on rare constants.
November 11th, 2005
In a previous blog post, I ranted a bit about how candidates can make themselves more attractive to employers. While I still am interested in things from a employer’s perspective, there are three critical consideration for a candidate: Do I want to do the role offered? Is there an opportunity to have an impact? Will I have the support and environment necessary to effectively do the job?
For each candidate, the consideration of each question will vary as well as its important. During all research and interaction with the organization, a candidate should structure her questions to build a “dossier” that addressees these concerns.
Do I want the role offered?
While a relatively simple question, it requires that a candidate has a clear sense of his goals and priorities. Timing is generally a key component, as different roles may be more or less crucial throughout the different stages of professional development.
Is there an opportunity to have an impact?
Ultimately, I think the most rewarding positions are ones in which there is an opportunity to have a significant impact to the organization. The organization benefits and the candidate usually does as well; the experience is generally better and prospective employers want to know what results were achieved. The scope and scale of this opportunity will usually vary with the position; a developer who introduces TDD to a group may help lower defect rates while a VP of Engineering has the opportunity to impact the company’s profitability. However, I think all candidates, regardless of the position, should be ambitious and consider the largest impact the role may have. Similarly, it is a big warning sign if the position being considered has little impact to the organization; there will generally be few resources, support, professional development, etc.
Will I have the support and environment necessary to effectively do the job?
This is a very open ended question and probably the most personal for a candidate. Part of the consideration is about organization support: autonomy, authority, budget, latitude, and counsel are some significant elements. Environment is quite personal. Some candidates like dynamic, fast paced environments and others prefer academic settings. For some people, co-workers that can be potential friends is significant. Some people need flexible working hours. Ultimately, an employee will be most effective in the environment best suited for her.
While these questions may seem obvious, I am always surprised by how few candidates (or so it appears) are trying to address these considerations. A key part of my evaluation of a candidate is an assessment of his evaluation of the position; I want candidates who are looking just as hard at my organization and I am looking at them. Also, you will notice that there is a certain bias in these considerations towards professional development and satisfaction. I’ve made no mention of practical considerations such as salary. While practical concerns need to be satisfied, I don’t think they should be prime motivators. This is not to say that elements such as salary aren’t important considerations, but I think it is harder to find the position and organization that matches well with the three described concerns than to find a position to satisfy practical issues.
October 19th, 2005
I have interviewed a lot of people over the years. (I’ve hired over 50 people, interviewed hundreds, phone screened many more and read thousands of resumes.) While I don’t claim to be any sort of recruiting expert, there are two things I can intelligently (as much as I can for anything else) speak of: trends in resumes / candidates and guidelines for being stronger on both.
The single most important fact to understand as a job seeker is: A prospective employer has no attention span whatsoever.
Your goal is to get the attention of the person who will actually decide to hire you and make clear why you are different from all the other candidates she comes across. Employers must sift through many resumes (there were times I would look at 100 resumes a day) and frequently cannot afford to spend a lot of time on a resume. Assume you must communicate in 10 seconds a reason that the employer should continue reading. Recruiters, internal or external, are not your target audience. While you may need get past them to reach the actual employer, never forget that your message should be targeting the actual employer.
While this sounds obvious, you are selling yourself to an employer. However, most resumes I see are not selling but reporting. Listing the details of each job, tool, language, API, or activity from your entire experience is not effective.
A general sales adage is a product should be made easy to buy, not simply easy to sell. When faced with buying something that is a “no-brainer”, there is no selling. A candidate should always consider the perspective of the employer; what would make it a “no-brainer” to hire me?
There are many different approaches and considerations for how a candidate can position herself as an obvious choice. Ultimately, I think it can be summarized by two questions (from the point of view of the employer): What can this candidate do for me? How does this help me? The first question is about the results the candidate can achieve or deliver. The second is about the match of these skills for the employer’s context.
The following are some guidelines for candidates:
Overall
- Have a 30 second elevator pitch.
- Know what your strengths are. Know what results you have achieved. What makes you special and stand out?
- Know your professional goals.
- What are your career plans? Where do you want to be in one year? Five years?
- Have a career plan.
- The goals are a strategic concern; your plan is tactical. You should be able to describe the role you want to perform, with detailed functions and responsibilities. Be able to describe an ideal work day, month, or year.
- Know your strengths, weaknesses and what affects them.
- More importantly than what you do well or poorly are the factors that enhance your skill set and those that detract from it. These are critical to selecting the right position.
Resume
A resume makes the first impression. It must get someone’s attention immediately and should answer the two most important questions above for the employer.
- 1 page only. No exceptions.
- I don’t read 7 pages resumes; if the person can’t be brief, it implies they are disorganized and cluttered. I will note one caveat to this: your resume should be one page. It is an introduction and summary. If there are details that are important to communicate, include an addendum or appendix (which should be never more than 2-3 pages) and make it clear that this is extra, not part of the resume. You can title it “Project Details” or something, but it should have it’s own header and not be confused as part of the resume.
- Lead with the elevator pitch.
- Your 30 second elevator pitch should be communicated immediately in a “Professional Profile” or similar section.
- Focus on results and achievements
- When I read that a candidate was on a project that tripled company revenues, I am interested to continue reading.
- Highlight technical qualifications
- Do not list every three letter acronym you know, even if you are an expert. Focus on your core strengths and the technologies you want to work with or that are relevant to the employer. If you know C++ but are looking for job doing Java, don’t mention you know C++.
Almost 100% of the resumes I read look the same. They all share the same subset of technical jargon references and technologies. There are many, many J2EE, .NET, C++, etc. developers. It is almost impossible to differentiate yourself in this category. Though a technical profile is obligatory, as a technical worker, it should be brief and contain highlights. I suggest putting it at the end of the resume.
The technical profile of a resume is the source of the largest mistakes in candidate positioning, so I will babble and rant about this a bit. I think the reason for these mistakes are:
- technical people focus on technology
- many recruiters encourage listing all your technical experience
- employers may screen based on the content of technical profile
The first point is probably pretty obvious and I won’t discuss it. The second is also common, though I discourage it. The last reason is the most significant: frequently resumes are so poor (or there are so many) that employers have little choice than to screen based on technical profiles. However, I think there are a few mistakes being made here. First, candidates try to keep as many options open as possible and this actually hurts them. Few people are experts in the large lists of technologies, languages and tools commonly listed on resumes. Even if you are, simplify. For example, if you’ve worked with many Web Service / SOAP technologies, write in your technical profile something like “SOAP / Web Services expert (worked with Axis, WebMethods, BlueTitan …) instead of listing the 20 different Web Service related frameworks you know. Alternatively, if your goal is to work on web services, only list the web service frameworks you know and omit the other items such as object relational mappers, databases, etc.
Listing too many elements in a technical profile has the risk of implying indecision and insecurity. Don’t list technologies you don’t really want to work with. Also, a candidate that doesn’t care what technologies they work with (when they are listing many, unrelated technologies) seems to have little career focus. Finally, it may convey insecurity in the candidate; it can make a candidate look desperate and thus undesirable. Less is more.
Phone Screens
In over 90% of the phone screens that I do, I know if I will pursue a candidate in the first two minutes of the phone screen. A phone screen is intended to serve answer one question: Should I invest the time to formally interview this candidate? Formal interviews are generally very expensive to conduct and therefore most organizations don’t do them lightly. There are a few things to keep in mind at this point.
- Research the company
- I hope this is obvious, but it amazes me how many people do not do this. Look at the companies website, competitor websites and industry websites for the employer. Learn as much as you can.
- Be prepared
- Know your own experience well, such that you can communicate succinctly. Learn how to be brief and focus on highlights. Interviewers will ask follow-up questions when interested and will appreciate brevity. Also, give thought to typical questions and know your answers. Such questions include: Why are you looking for a job? What is the ideal job, role, organization? What is the most important thing I should know about you as a candidate? Tell me about an interesting technical or team challenge and how you resolved it. What was the best project your were on? Worst? Why?
- Interview the employer organization
- You should look as closely at the employer company as they are looking at you. Good candidates take their professional development very seriously and are engaged in knowing about the company, its culture, its business model, future, history, etc.
If you are looking to work for me…
This entire blog entry is biased about making a candidate attractive to me, as an employer, but there are a few other items of note:
- Communication Skills are Key
- Being able to communicate clearly, effectively and succinctly is critical. It is vital to working in an agile environment and poor communication is one of the core reasons I don’t pursue candidates.
- Must be a Team Player
- Agile Processes emphasize collaboration. You must thrive in a team environment.
- Be Honest
- I generally know when a candidate is weak in a particularly area, unsure or blatantly lying. I prefer and respect an honest and direct “I don’t know” rather than trying to dance around ignorance.
- Be Committed to Projects
- I am looking for people who will take ownership of the result and do what they can to achieve it. I want candidates who take the outcome as a personal reflection of their abilities and work ethic. I don’t ask for late nights or weekends but respect people who sacrifice when they need to make sure something is done.
Conclusion
Employers hire people, not cogs. A candidate should never forget that an employer is investing in a person and should do everything they can to position their value as an individual to an organization. If you are seen as just a Java developer who knows SOAP, you are easily replaced. Great employees, however, are assets to companies and rare.
September 23rd, 2005
In a recent blog post I discussed the “Market / Application Gap.” In order for a software product to succeed, it must align with the user’s needs. The faster an organization can bridge the gap between the market and the actual application, the greater its competitive advantage and its opportunity for success.
At the start of a project, some effort is given to requirements discovery and this effort varies with software methodology. From here requirements are communicated to engineers who in turn produce an application that reflects the requirement. With agile software development there is an emphasis on taking action as early as possible; waterfall tends to focus on a more complete sets of requirements, designs and test plans (read: more time). With my “Market / Application Gap” diagram in mind, the process looks something like this (order of events, with respect to information flow / deliverables, are numbered):
This cycle represents an iteration: from requirements to delivery of an application. While much has been discussed about the iterative nature of agile development, the truth is that every software process uses iterations. The only question is the length of the iteration. An agile project might have an iteration length of weeks to months; waterfall can have iterations that last years. (Remember, I am defining an iteration in this context as the delivery of an application to the end user.)
At the end of an iteration, the overall gap between the market and the application either grows or shrinks. If you have an iteration cycle of a few months (at most), you can recover if you missed the needs of your market. When you have an iteration cycle of years, you may not get a second chance if you missed the mark. This is one of the core reasons why agile development makes more sense than waterfall: from a risk management point of view it is foolish to take a “big bang” approach where you have few opportunities to satisfy your market.
If the goal is to close the gap between the related participants in the “Market / Application Gap”, the quicker you can complete an iteration, the quicker you possibly close the gap. Many successful software products simply work the way users expect; unless you’re omnipotent, this requires some trial and error and probably some mistakes. Users will tolerate some mistakes, but won’t wait around long. Long iterations can result in a lost audience, which spells doom for commercial software. In my years of software development, I’ve seen a variety of software processes, many of which claim to encourage iterations and quick feedback. Agile Software Development is the only approach I’ve seen deliver on this promise in actual practice.
September 2nd, 2005
Previous Posts
