Agile Development Methodologies: What are they and why is everybody in the software world talking about them? Karen McKenna TES APS Project Office Email: Karen.e.mckenna@telus.com
Context of research Understand Industry trends Research!!! Food for thought One point of view Controversial Open mind
Agenda Background Information: Terminology Overview What is Agile Methodology? What is Iterative Development? Agile vs. Defined which wins? Methodology Landscape Agile Methodologies: XP DSDM SCRUM What s happening in the industry? Implementing a new methodology
Terminology Process describes what to do Methodology describes how to do it Agile Methodology a.k.a. = Lightweight methodology Defined Process Methodology a.k.a.= Heavyweight methodology ISD = Iterative Software Development (subset of newer methodologies)
Focus on: Defined Methodologies Formal processes E.g. SUMMIT, Accenture Method/1, Navigator, RUP Focus is on the process steps to follow
Agile Methodologies Focus on: Improved time to market Increased collaboration around team members Strong user participation Highly iterative releases (ISD) Focused on the coding process and code, not documentation Deliver incremental applications without the necessity of satisfying the complete set of requirements Best suited for: Innovation based products New product delivery
Agile Manifesto Principles of Agile Development: Individuals and interactions are valued over processes and tools Working software is valued over comprehensive documentation Customer collaboration is valued over contract negotiation Responding to change is valued over following a plan
Agile Methodologies Types of Agile Methodologies: XP (Extreme Programming) SCRUM DSDM (RAD)
Are you ready?
Iterative Development (ISD) Philosophy: Change is the rule not the exception It is impossible to predict what activities the project needs to do in order to complete the job so we need to base our planning upon uncertainty Sounds simple but it is a big shift in mindset. Iteration: the controlled reworking of a system to meet refined requirements or make improvements A.k.a. evolutionary development You need full time commitment from developers and customers.
Waterfall vs Iterative Waterfall Emphasis on specs get them right, complete, signed off Assumptions: Customer knows what he wants Requirements are frozen (changes are the exception) Phase Reviews are used as control and feedback point Characteristics of successful projects: Stable requirements Stable environments Focus on the big picture One, monolithic delivery Risk is postponed to the end integration hell Makes it easy on the manager and difficult for the engineering team Iterative Software comes first Assumptions: Customer can not express exactly what he wants Requirements will change Reviews are done continuously for control and feedback Characteristics of successful projects: Fast and continuous customer feedback Floating targets for the product Focus is on most important features Frequent releases Risk is assessed and validated early and often More aligned with how software engineers work (teams have a 5:1 ratio of engineers to managers)
Are you ready? If you decide to adopt an ISD approach, you need to be ready for a new mindset. If you love the security of having a long, detailed project plan, this may not work in your world. You have to trust: That you will get to your target faster, by accepting that you don t exactly know how to get there That you will utilize what you learn on your way, and that this learning and openness to change will lead you to a better result Can be a difficult transition for PMs: article from Rational Software guru Philippe Kruchten in handout
A View on Detailed Long Term Planning Forget about anything you ever learned about planning. Creating an end-to-end project plan is a waste of time (except to examine schedule and resources). The plan is outdated before you reach the first iteration : PMs have to understand their role is to educate and explain the notion of iterative planning: E.g. family vacation to Disneyland / Legoland - the overall route is etched out, but the only detailed instructions needed at the start is what to do once you get to Anaheim and how to get to the hotel. Carrying a big notebook and planning the exact details of what ride you are going to go on the second day at 10:00 is unnecessary. You may find that there is a long line up at one attraction and you need to modify your schedule.
Do you need to move to Iterative? Depends on: Stability of the requirements Your relationship with the customer
Stability of the Requirements It may not be worth the effort if: Your customer delivers a complete detailed requirement specification (has this ever happened?) Your customer is happy if he receives exactly what was written in the requirements document Your customer at the end will accept that delivery is late, because getting the full functionality is more important
Stability of the Requirements Iterative is worth the effort if: The requirements will change substantially Delivering on time is essential It is not possible to specify the requirements in enough detail up front You have an unstable environment
Customer Relationship ISD is a very different relationship between the development team and the customer. Both groups need to agree to use ISD and embrace the the possibilities it provides. If your client is only willing to deal with standard contracts that document everything imaginable, and if you don t deliver to the letter of the agreement it s hello Mr. Lawyer I ve got a problem!! then you need to be tread carefully. You may get business value from some of the practices, but you won t get the full advantage. If the customer is agreeable to taking more responsibility and taking active part in the development activities, your mutual benefit from ISD will increase.
Funding Model: a critical component Many projects are pushed into bidding for a contractual development far too early, somewhere in the middle of inception. In iterative development, the best point in time for all parities to do such bidding is at the LCA milestone (end of elaboration). A two step bidding process is preferred. Source: Rational Software (Kruchten)
Methodology Landscape Process Rich (including PM, CM, etc.) PWC Summit, Partial life-cycle Rational Navigator* Full life-cycle XP, SCRUM, Microsoft MSF DSDM Limited Process Support
Agile vs Defined: Which wins? Agile process (eg. XP, DSDM) vs. Defined process (egg. RUP, PWC Summit) Agile assumes the development process is unpredictable and chaotic Defined process argues that mayhem is due to the immaturity of the organization (SEI CMM) To expedite project delivery in turbulent times there is a need to be open to different approaches. However, the choice between a defined process and an agile process must be implemented on a project level
Some Agile Methodologies Extreme Programming (XP) DSDM SCRUM
XP (Extreme Programming) Originated due to concerns that software development life-cycle methodologies were becoming overly complex and too focused on documentation oriented deliverables, rather than actual code; Many shops failed to implement formal methodologies There was a need for a methodology to survive under the pressure of faster, better, cheaper environment Looking for balance of formalized hacking and unshackled process
XP (Extreme Programming) Four Key Values: Communication Feedback Simplicity Courage Project must be quantified by four variables: scope, resources, time and quality. Management can dictate three of the four project variables; development always gets the fourth
Twelve XP Practices 1. Planning Game The customer picks the features wanted/needed. Programmers estimate each feature, the customer chooses which feature is done in which iteration 2. 40 hour week Team members should not be expected to regularly put in overtime 3. Pair Programming Two people work on the same piece of code (and the same workstation), one writing and one reviewing theory that this increases code quality and reduces errors This is the most controversial practice
Twelve XP Practices
XP Practices 4. Collective ownership of code: Code base is shared but the team, and all are expected to have both depth and breadth of knowledge 5. Continuous Integration: No code is allowed to sit off in a branch and remain detached from the core code base for too long. Integration is thus streamlined. 6. Simple Design: use the simplest thing that could possibly work Build the minimum necessary 7. Coding Standards Teams must agree on and adhere to coding standards that are established upfront.
XP Practices 8. Test first Test plans are defined before code is even written. The emphasis on quality is built in. 9. Aggressive Refactoring: Since design is minimized this is required Improve the design of the project throughout the development life cycle 10. On-site customer Customer is an active part of the team, setting priorities, setting and clarifying requirements providing the detail needed to implement terse user stories
XP Practices 11. Small releases Frequent and small releases are designed to enable an XP team to get to production early. The team gets more feedback and ensures the resulting system is what the customer wants 12. Metaphor XP doesn t have the formal notion of an architecture, but defines a common naming convention and description of the system that establishes a shared metaphor of the system in all of the developer minds.(the big picture)
XP Industry adoption rate Adoption is on the rise. Industry adoption of practice: Higher usage in high-tech sector To date, hasn t been terribly successful in traditional IT shops due to: Inability to use on large applications The focus is on development. Project management, change management, requirements management processes must be added on to the methods to create a full SDLC. cultural issues
Projects suited for XP When requirements are constantly changing or not fully known up front Small project teams working from the same site (2 10 people) Not suited for large,complex projects You must be able to create automated unit and functional tests Risky projects with dynamic requirements
DSDM Dynamic Systems Development Method Arose from frustrations of users and IT alike with methods that were seen as inappropriate for a fast moving business environment, forced users to fix their requirements in concrete early in the cycle and did not allow for rapid iterative delivery. Intended to be different from the classical sequential (or "Waterfall") methods for application development.
DSDM Dynamic Systems Development Method Standard devised by the DSDM Consortium, a British organization focused on a RAD approach that increasingly is used for iterative or "agile" development Industry representatives realized the need for a standard RAD framework (I.e. grassroots organization) Started getting industry attention in 1995
DSDM TENDS TO WORK WHEN The application will be run standalone. Major use can be made of preexisting class libraries (APIs). Performance is not critical. Project scope (macro-schedule) is constrained. Reliability is not critical. The required technology is more than a year old.
What s Different About DSDM It s based on the 80/20 principle - nothing is built perfectly first time It s an iterative development approach based on prototyping It assumes requirements change as understanding increases It concentrates on business needs, not development activities (business driven not IT led) The result is quality systems that are fit for purpose
DSDM Lifecycle Phases: Pre-Project Feasibility Study Business Study Functional Model Iteration Design and Build Iteration Implementation Post-Project sequential Iterative and incremental How the three later phases overlap and merge is left to a particular project to decide.
DSDM Lifecycle
Benefits Of Using DSDM The users are more likely to claim ownership of the solution The risk of building the wrong solution is greatly reduced The final solution is more likely to meet the users real business requirements The users will be trained prior to deployment The implementation of the business solution is more likely to go smoothly
Survey says.. DSDM claims that some organizations have experienced: Five fold reduction in delivery time Team size reduced by half 10% increase in success rate
DSDM Phases Feasibility Study is DSDM is the right approach for the project? assessment of the likely costs and of the technical feasibility of delivering a system to solve the business problem. no more than a few weeks
DSDM Phases Business Study focus on the business processes affected and their information needs activity is strongly collaborative: series of facilitated workshops attended by knowledgeable and empowered staff who can quickly pool their knowledge and gain consensus as to the priorities of the development.
DSDM Principles 1. Active user involvement is imperative 2. The team must be empowered to make decisions. 3. The focus is on frequent delivery of products. 4. Fitness for business purpose is the essential criterion for acceptance of deliverables. 5. Iterative and incremental development is necessary to converge on an accurate business solution. 6. All changes during development are reversible.
DSDM Principles 7. Requirements are base lined at a high level. 8. Testing is integrated throughout the lifecycle. 9. Collaboration and cooperation between all stakeholders is essential
DSDM - Time boxes Mechanism for handling flexibility of requirements Each time box has a fixed end date and a prioritized set of requirements assigned to it. Mandatory Nice to have room for maneuver when things don't go perfectly to plan The prioritization of the requirements throughout the time box are checked and possibly reassigned using the MoSCoW Rules.
Prioritizing Requirements _ MOSCOW rules Provide the basis on which decisions are made Must Haves fundamental to the projects success o Should Haves important but the projects success does not rely on these Could Haves can easily be left out without impacting on the project o Won't Have this time round can be left out this time and done at a later date All the priorities are reviewed throughout the project to ensure they are still valid. It is also essential that not everything within a project or a time box is mandatory.
SCRUM SCRUM's key assertion: software development is inherently a chaotic, complicated, roughly sequential set of activities that can only be loosely described as a process.
SCRUM Expresses requirements in the form of a prioritized list called a Backlog. This list is assumed to change on a daily basis. Implementation iterations are called Sprints: two- to four-week full (analysis, design and implementation) iterations with one of the inputs being a list of requirements from the Backlog. The input requirements are not allowed to change during the Sprint. Key feature: daily, short, cross-functional meeting to monitor blocks, possible changed priorities and progress.
Advantages of SCRUM By assuming requirements are in flux, it avoids the analysis paralysis of complete requirements specification. Can respond to external changes of requirements, changed delivery dates, changed functionality and changes in resource availability. Developers are encouraged to be creative within the boundaries of a Sprint.
Comments on SCRUM SCRUM has been deeply influential on other lightweight methodologies but has been pushed aside by XP. There is concern that the methodology does not scale beyond small teams and small projects.
What is happening in the industry? Adoption of Agile processes is on the rise. Expected to have some presence in most IT shops in the next few years. Reuse is becoming more important: You need a solid process to be able to do it. Methodology vendors in the market are stabilizing Key vendors: Rational, CA, ASG, Aonix, Fujitsu.
Methodology Implementation Methodology implementations are risky, challenging and very costly. Support must come from the top CIO has to support it. If you get support higher up it will help considerably Bacon and egg story chicken involved, pig committed! Dedicated staff for implementation, support, training etc. Start with a trial project and roll out slowly Do not attempt to introduce a new methodologies midstream in a project Measure your success (numbers, metrics)