Managing a Large Agile Software Engineering Organization. Paul Beavers BMC Software

Transcription

1 Managing a Large Agile Software Engineering Organization Paul Beavers BMC Software

2 Managing a Large Agile Software Engineering Organization Who am I? Paul Beavers Software Engineering Director Large (>200) global organization spread across four international locations Developing software for 20 years Learning something new every day

3 The Journey The results In the beginning... Immersed in Agile Harsh realities Transformation Reflections Continuing the dialogue

4 The Journey The results In the beginning... Immersed in Agile Harsh realities Transformation Reflections Continuing the dialogue

5 Managing a Large Agile Software Engineering Organization The Results Four successful major releases Renewed confidence in the engineering team s ability execute Excitement about the quality and quantity of features being delivered New sense of ownership across >25 scrum teams Challenges addressed as they arise

6 The BMC/SQM/Cutter Consortium Study Project Demographics * Two major BMC Software releases were assessed (BMC Performance Manager R2.3 and R2.4). Schedules ranged from months, full-time equivalent staff (FTEs), delivering approximately 1,450 story cards, comprised of ~1.4 million lines of new and modified code. Both releases utilized the Agile SCRUM methodology put into place during at BMC Software. *Content provided by Cutter Consortium

7 Industry Data from the QSM SLIM-Metrics Database* Spans 20+ years Large heterogeneous database contains over 7,000+ projects Represents over 685+ million SLOC, 7+ million function points, over 600 languages, from 500+ organizations in 18 countries Adding projects/year *Content provided by Cutter Consortium

8 Productivity Index (PI) Assessment * Productivity Index First put forth by the pioneering metrics research by Larry Putnam An objective measure of the efficiency that is achieved in building a software product Encompasses all of the processes that influence the performance of a team given the complexity of their task and the development environment Calculating the PI requires three inputs, namely, size (such as Function Points, Lines of Code, modules, objects, etc.), time and effort The equation that calculates the Productivity Index takes the basic form: Quantity of Function = Process Productivity * Effort * Schedule This can be re-arranged to solve for Process Productivity as: Process Productivity = Quantity of Function Effort * Schedule 8 *Content provided by Cutter Consortium Copyright 7/11/2008 BMC Software, Inc.

9 Benchmark Process** Method Conducted on-site interviews on both releases Gathered industry standard core metrics for elapsed time, effort, size*, and defects Benchmarked the results, calculated performance values, and compared them to the QSM database Assessed schedule performance, FTE staffing levels, effort, defects, and productivity values for the requirements (Story) and Main Build development phases * Iterations, stories, and the resultant added/changed code (predominantly Java) **Content provided by Cutter Consortium

10 Input into SLIM-DataManager * Size Defects Time Effort *Content provided by Cutter Consortium

11 Example Productivity Index Calculation * Rel 2.4 PI = SIZE PI = TIME EFFORT * Size = 526 Stories/569k LOC = 27.8 Effort = 381 Person-Months Time = 4.5 Months (Build) *Content provided by Cutter Consortium

12 Trendline Assessment Schedule * Build Phase Time BMC QSM 2005 business 1 Sigma line style 1. Much faster than the industry trend. >95 th percentile. 2. Build phase was 4-5 months, versus more than a year for industry months 10 Months Release 2.4 Release months Faster Story Cards 1,000 *Content provided by Cutter Consortium

13 Trend line Assessment Schedule Agile Sample * BMC Agile Companies Build Phase Time 1. Blue circles are other Agile development projects from a sample of 4 organizations using XP and SCRUM. 2. Agile projects are faster as a whole. 100 QSM 2005 business 1 Sigma line style 10 Months Release 2.3 Release 2.4 Faster ,000 Story Cards *Content provided by Cutter Consortium

14 Trendline Assessment Quality * BMC Bugs/Defects (Hardening Iterations) 1. Defects at normal levels in spite of fast speed 10,000 QSM 2005 business 1 Sigma line style Release 2.4 Release 2.3 1,000 Bugs/Defe ects Stories 1,000 *Content provided by Cutter Consortium

15 Productivity Index Assessment * BMC Agile Companies Productivity Index 1. Agile projects as a whole tend to exhibit higher PIs Release 2.4 Release QSM 2005 business 1 Sigma line style Productivity Index (PI) ,000 Story Cards *Content provided by Cutter Consortium 0

16 Productivity Index (PI) Industry values by application type * Business Scientific System Process Control Telecommunications Command and Control Real Time Avionics Microcode BMC Software, PI=27! Information Engineering Real Time Productivity Index (PI) w/ ±1 Standard Deviation *Content provided by Cutter Consortium

17 The Journey The results In the beginning... Immersed in Agile Harsh realities Transformation Reflections Continuing the dialogue

18 In the beginning Business requirements Market excitement Single simplified integrated product line Time to market Features and functions Quality Product reputation Internal and external consumer confidence Opposing forces Waterfall legacy Fixed scope, fixed time, shrinking budgets Large installed base Large organization (12 engineering teams) Geographically disperse (Austin, Houston, Israel, India) Most engineers have between 5 and 20 years experience Tens of thousands of lines of existing code

19 The Journey The results In the beginning... Immersed in Agile Harsh realities Transformation Reflections Continuing the dialogue

20 Managing a Large Agile Software Engineering Organization A New Approach Introduction to scrum Consultants hired to coach, mentor, and establish Agile teams Staged implementation across engineering teams driven by lower level management Progressive proliferation of scrum Continuous evaluation and adjustment of agile practices Manage integrated product sets via scrum of scrums (SOS)

21 The Journey The results In the beginning... Immersed in Agile Harsh realities Transformation Reflections Continuing the dialogue

22 Reality Sets In Challenges Managing and meeting consumer expectations Lack of visibility into progress on features and functions Questionable quality General morale of the teams Teams questioning the methodology Increasing tension Increasing burnout Critical processes not working Team Dynamics Traditional management styles Lack of influence Cognitive dissonance Working for the metric Group Think Suppressed intuition Loss of creativity Organizational structure Internal competition Conflicting goals and priorities Lack of coordination between scrum teams, chaotic

23 Reality Sets In Traditional Measurement Quantitative Metrics Percent project complete Percent feature complete Defect arrival and closure Qualitative Metrics Rumors Hall Talk Fear and uncertainty False Sense of Security Working for the metric Lack of clear tie from metrics to business results Recognizing teams for good metrics and not good results Metric looks good but the product does not Overreaction Confusion Metrics look good Intuition indicates something different

24 The Journey The results In the beginning... Immersed in Agile Harsh realities Transformation Reflections Continuing the dialogue

25 Agile Thought Leadership Therapy sessions with leading Agile thought leaders Ongoing gdialogue regarding gsuccesses and challenges Organizational Structure Requirements management process Critical technical processes Most important and most humbling Management team does not have the answers they should have the questions Hands-on people are the ones who know Is it going well? How is the quality? What needs to improve? What is working well?

26 What does better look like? Measurement Aligned with desired d results Used to make decisions regarding risks and mitigation Qualitative Fist of Five Team confidence levels Quantitative Meaningful software complete (burn down) Iteration defect leakage Feature and defect backlogs Percentage of tests automated Product productivity index Leadership Trust the team Allow the team to influence Empower the team for success Expect and allow for failure Remove roadblocks Insulate the team from unnecessary pressures What is preventing the team from being able to design, build, test, and automate within an iteration?

27 Cross Scrum Coordination Scrum of Scrums Standing meeting of representatives from each of the scrum teams contributing to a given release Membership includes those who need to be involved Open membership team members decide if there is value in attending Regular intervals As frequent as necessary Weekly Daily As often as twice daily Simple Agenda Objectives Remove and prevent roadblocks Ensure coordination is occurring Sample Agenda Each Scrum answers the questions What has your team done since we last met? What will your team do before we meet again? Is anything slowing your team down or getting in their way? Are you about to put something in another team s way?

28 Managing a Large Agile Software Engineering Organization The Experience Change in management philosophy Evaluate And Improve Repeatable release cadence Applied experience First agile release Agile challenges Critical product issues New product strategy Waterfall challenges Agile (Scrum) Time

29 The Journey The results In the beginning... Immersed in Agile Harsh realities Transformation Reflections Continuing the dialogue

30 Reflections Establish a simple organizational structure to enable Agile development. The reporting relationships should be logical and understandable with direct lines of communication. Limit exceptions to the model which defines the functional structure. Have confidence in the engineering team and empower them for success. Provide each team with the necessary requirements and prioritization and allow them to manage their backlogs.

31 Reflections Trust your intuition and act quickly. Avoid the desire to acquire too much information before making a decision. Let the risk associated with a wrong choice drive the effort put into making the decision. Pay close attention to qualitative measurement and let go of Pay close attention to qualitative measurement and let go of traditional quantitative metrics. The intuition of the team members is infinitely valuable when determining quality levels and release status.

32 Reflections Stay involved as an active team member. There is immeasurable value for the information gained when managers stop in to a daily standup meeting, an iteration planning session, or iteration demo and participate as if they were an active team member. It is important to remember to behave as a team member and not direct the process. Manage and prioritize software requirements in a way which is conducive to Agile software development. Follow the guidelines for writing good software requirements (Independent, Negotiable, Valuable, Estimatable, Small, Testable).

33 Reflections Invest in an automated nightly build and verification process. Avoid under estimating the importance of having regular successful product builds and the effort required to produce them. Maintain release readiness. Having a market releasable product at the end of iteration should be the goal. Defect prioritization should focus on fixing the defects that would prevent a product from releasing first. Drive toward full regression testing within the iteration using automated testing techniques.

34 Reflections Trust the methodology. When projects get chaotic, avoid the tendency to revert to old style software development. Allow the Agile process to work to address key issues.

35 Leading from the front Think once, think twice, think thrice whether you are really, really, really ready to empower in the full sense of the word. Empowerment is like an acid test for better or worse it will give you your defining moments. Israel Gat, BMC Software

36 The Journey The results In the beginning... Immersed in Agile Harsh realities Transformation Reflections Continuing the dialogue

37 Continuing the dialogue Bloghttp://every2weeks.wordpress.com/ Its not enough to simply change what you do, you must change the Its not enough to simply change what you do, you must change the way you think