Agile software development processes have received much attention from the software development industry in the last decade. The goal of agile processes is to focus the importance of people as primary contributors of the project and reduce the administrative overhead of producing working code for the stakeholders of the project. This paper explores some of the explicit and implied constraints of agile software development processes. It focuses on several common practices of agile processes, particularly those that might limit their adoption by large and geographically distributed organizations. This paper makes recommendations to reduce the barriers to adoption of agile processes by these types of organizations. It attempts to answer questions such as:
- Is it possible for a large organization with many established business and development processes to incrementally adopt an agile process?
- Is it possible to adopt agile development processes to work for many individuals who are physically isolated, such as work-at-home software developers?
- Is it possible to adopt agile development processes to work for a large team, divided into many sub-teams that are geographically distributed and possibly working in different time zones?
Extreme Programming is probably one of the most recognized agile software development process today. It was introduced in the late 1990s by Kent Beck and eventually published as a book (Beck, 2005). Beck’s approach documented the values, principles and practices necessary too deliver lower defect, working software with less formal process and more focus on the skills of people and community that produces it. Extreme Programming is targeted to small, collocated teams of about twelve people. Other proponents of agile software development processes understood the increasing interest in their approaches by the software industry and followed with the Manifesto for Agile Software Development. The contributors of the Manifesto were the creators of many different agile, iterative and incremental software development processes. Their goal was to unify principles they shared in common. The work was authored by “[…] representatives from Extreme Programming, SCRUM, DSDM, Adaptive Software Development, Crystal, Feature-Driven Development, Pragmatic Programming, and others […]” (Manifesto, 2001).
Beck and Andres (2005) present the primary practices of Extreme Programming in their book. Two practices stand out as a limitation of scaling Extreme Programming to teams in multiple locations, or even work-at-home employees. They are Sit Together and Pair Programming. Sit Together is a practice that encourages the team to work in a unified area, such as a large, open room that promotes easy communication. Pair Programming is a technique where two developers sit together at a single workstation and take turns designing and writing code. As one developer is writing code, the other is observing, asking questions and offering suggestions as the current piece of work progresses. The goal of these two practices is to lower the defect rate through a constantly available communication and collaboration of developers sharing the same physical space. Beck and Andres (2005) also discuss the importance of team size in a project that uses Extreme Programming. They recommend a team size of about twelve people. The reason for this size has as much to do with coordination of development activities as it does with psychological needs of being a part of a team. The larger a team grows, the less personal the connections between team members become. Faces are more difficult to remember and communication among all members gravitates toward infrequency. These challenges with team size become amplified with work-at-home software developers who may only be in the physical presence of other members of the team a few times a year at specific events such as all-hands meetings.
Active and regular communication is a requirement with agile software development. Ramesh (2006) describes the perceived advantages of teams distributed across time zones and continuous development, e.g. as one team ends for the day and goes to bed another is coming to work to pick up where the last left off. However, there is actually a communication disconnection between the geographically distributed team in this situation, and the teams are forced into a mode of asynchronous communication, potentially slowing down progress. This problem relates to two principles of the Manifesto for Agile Software Development (2001) that presents a challenge to geographically distributed teams. The first is “Business people and developers must work together daily throughout the project.” The second is “The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.” Both principles are related to communication among developers, management, stakeholders and users of the project.
Lindvall (2004) points out that incremental adoption of agile practices into an existing large organization can be challenging. An existing organization typically has the expectation that existing business and development processes are followed regardless of project size and the process used. Educating those outside of the agile pilot project and resetting their expectations for following the established processes can create tension. A specific example is that development in agile-driven projects usually starts with a subset of the requirements set. This is a quality of agile development processes and has to do with working on what is understood to be the goal of the project today. As working builds are created and delivered to stakeholders the requirements set can be appended and refined until there is agreement that a reasonable goal has been established. Murthi (2002) documents a case of using Extreme Programming on a 50-person development project and cites the ease of starting early with a partial requirements set, and using the subsequent working results for two goals: show stakeholders working software to build confidence in the development team and giving stakeholders something to help refine their own understanding of their needs.
Incrementally developed requirements, constantly refined budgeting and burn rate of finances that are typical of agile development process management can present a unique challenge to a project that is completely or partially outsourced. Cusumano (2008) details the need for an iterative contract between the customer and outsourcing provider. A fixed-price contract can be nearly impossible to design when agile development processes are in use by either party. Boehm (2005) also discusses the problem of using agile processes within the realm of contracting to the private and public sector. Problems can be encountered when measuring progress of a contract’s completion. As a consumer following an agile process, the requirements can remain a moving target well into the project’s life cycle. As a provider following agile development processes, it can become nearly impossible to provide final system architectural details early in the life cycle to the consumer for review. Boehm also points out the difficulties to overcome by providers utilizing agile processes when seeking certification for CMMI and ISO-related international standards.
The barriers of agile development process adoption by a large or geographically distributed organization can be reduced by a combination of two approaches. The first approach is the application of tooling and technologies to support the practices of agile software development that scale to an organization’s needs. The second approach is to continuously refine the practices in conflict with the organization’s existing mode of operation over time.
An example of practice refinement through technology adoption is The Sit Together and Pair Programming practices from Extreme Programming, and working together daily and face-to-face interaction among customers and developers as recommended principles of the Manifesto of Agile Software Development. These practices and principles are the most obvious barrier to adoption of pure agile development processes within a large or geographically distributed team. The essence of the Sit Together practice is to provide a means to communicate at-will among team members. Technologies that help support this practice in distributed environments include instant messaging systems that can provide a mechanism for short question and answer sessions for two or more participants in the project at once. Longer conversations among the team can be supported through VOIP solutions, reservation-less teleconference solutions, Skype and XMPP-based messaging solutions that can allow several team members at a time impromptu contact and discussion opportunities for project issues. Speakerphones allow collocated sub-teams to participate in conversations about the project across geographic locations. In all the examples cited, full-duplex voice communication is essential for effective discussion among several team members at once. This type of communication allows the audio channels to work in both directions simultaneously, which means someone can talk and interrupt someone speaking as they could when they are in person. Many inexpensive speakerphones are half duplex. These types of devices block the receiving audio channel when the person is speaking. Someone wanting to stop the speaker to clarify a point is unable to do so until the person speaking pauses. Background noise, such as a loud computer fan or air conditioner can cause similar problems for half-duplex communication systems. Pair Programming can be performed through a combination of voice communication and desktop screen sharing technology. Individuals working within the same network or virtual private network can use solutions like Microsoft NetMeeting or Virtual Network Computing (VNC) to share, view and work within each other’s development environment and perform pair programming over any distance.
Web-based and wide-area-network tooling to support the incremental development and tracking of plans, requirements and defects is available from several vendors such as IBM and Rally Software Development Corporation. Gamma (2005) presented The Eclipse Way at EclipseCon several years ago. The motivation behind his presentation was the many requests he received from users of the Eclipse environment to understand how a team distributed throughout the world could continue to release as planned and with a low defect rate. The Eclipse Foundation has a centralized data center in Canada for several of its activities including continuous integration and automated testing of nightly builds. The build and testing process of the Eclipse environment is fully automated for each platform it supports. Additionally, end-users are encouraged to install and use nightly builds after they pass the automated suite of tests.
Other barriers to adopting agile development processes cannot be solved with tooling alone. Ramesh (2006) found that the solution to working across multiple time zones is to synchronize some meetings, and rotate the time of the meeting so that each group takes turns in suffering from an extraordinarily early or late meeting so that everyone on the project can communicate live. Solving the opposing forces in contract negotiating requires creativity. Boehm (2005) recommends disbursing “[…] payments upon delivery of working running software or demonstration of progress rather than completion of artifacts or reviews.” According to Boehm there is not yet a well-defined compatible solution to agility in process and certification of ISO or CMMI related certifications. Lindvall (2004) concluded that adoption of agile development processes by large organizations is best accomplished through hybrid integration with the existing processes, particularly the established quality processes. With this approach, the existing quality processes can be used to measure the effectiveness of the agile software development process under pilot.
This paper described several of the qualities shared by different agile software development processes. It focused on those aspects that potentially limit agile process adoption by large and geographically distributed organizations. The recommendations made in this paper include technology solutions to improve collaboration and communication among distributed developers and consumers of the project. The technology considerations also help alleviate management concerns such as incremental planning and budgeting of agile projects. Recommendations were also provided for large organizations with established processes and approaches pilot projects utilizing agile development can take to leverage those processes to demonstrate their value. It is possible to adopt agile software development processes for large and geographically distributed organizations. Adoption requires thoughtful and careful application, integration and refinement of the practices at the core of these agile processes for a successful outcome.
Beck, K., Andres, C. (2005). Extreme Programming Explained. Second Edition. Copyright 2005, Pearson Education, Inc.
Boehm, B., Turner, R. (2005). Management Challenges to Implementing Agile Processes in Traditional Development Organizations. IEEE Software. 0740-7459⁄05.
Cusumano, M.A. (2008). Managing Software Development in Globally Distributed Teams. Communications of the ACM. February 2008/Vol. 51, No. 2.
Gamma, E., Wiegand, J. (2005). Presentation: The Eclipse Way, Processes That Adapt. EclipseCon 2005. Copyright 2005 by International Business Machines.
Leffingwell, D. (2007). Scaling Software Agility: Best Practices for Large Enterprises. Copyright 2007 by Pearson Education, Inc.
Lindvall, M., Muthig, D., Dagnino, A., Wallin, C., Stupperich, M., Kiefer, D., May, J., Kahkonen, T. (2004). IEEE Computer. 0018-9162⁄04.
Manifesto. (2001). Manifesto for Agile Software Development. Retrieved 2 October 2008 from http://agilemanifesto.org/.
Murthi, S. (2002). Scaling Agile Methods - Can Extreme Programming Work for Large Projects? www.newarchitectmag.com. October 2002.
Ramesh, B., Cao, L., Mohan, K., Xu, P. (2006). Can Distributed Software Development Be Agile? Communications of the ACM. October 2006/Vol. 49, No. 10.