Agile SCRUM in Systems Engineering A Practical Application Author Paul Wheway, Principal Systems Engineer, Thales UK. Paul.wheway@uk.thalesgroup.com Categorisation Accessibility Practitioner Application Case Study Topic Agile Systems Abstract Agile methodologies have been used extensively in software development now for a number of years; however, applications in Systems Engineering have been limited. The complex nature of typical Systems Engineering projects is often seen to be a barrier to adoption of Agile, the traditional V-lifecycle and formal contract milestones do not gel well with Agile working. Furthermore, issues exist in the interface between systems and software engineers, including the misalignment of processes and lifecycles. To try and mitigate these issues and investigate the challenges, the Systems Engineering elements of a Software dominant project were approached with an Agile / SCRUM methodology. The application of the principles of Agile to systems engineering are explored herein through real world experience on a live project. As a part of this approach, co-engineering is a theme that arises and applies to Agile working. This co-engineering helps to develop solutions that are both buildable and understood by all of the team. The approach taken, with a phased introduction of this method of working, lead to positive results and benefits being realised both in terms of demonstrating that an Agile SCRUM approach can work for Systems Engineering in these types of projects and in breaking down barriers between different disciplines. This paper outlines the approach taken for adoption, outcomes, and the lessons learnt of this work. The paper shows how the Agile SCRUM approach can be applied to software dominant projects in a practical manner and how this could be extended further to projects where software is only one element.
Introduction This paper describes a practical application of Agile SCRUM to Systems Engineering. The process was implemented on an existing project that was already using SCRUM for software development. The team consists of software and systems engineers. The aim of this paper is to describe the practical approach taken to implementing Agile Systems Engineering and some of the issues that were encountered and the outputs of this. The goal of implementing SCRUM was to improve the output of the team as a whole and to remove perceived barriers between systems engineering and software engineering. The barriers that existed were around roles being responsible for certain elements (e.g. development team, test team, design team etc.), a reluctance to broaden the responsibilities and to understand the impacts on others, and a tendency to throw items over the fence between teams. Issues that existed in the project were: No buy-in to requirements and design from software team Systems engineers specifying a system that the software team couldn t implement Missed requirements and incorrect implementation Large number of defects found late in test cycle Builds delayed into test cycle Defects not understood by software engineers It was apparent that the large majority of these issues were due to the lack of communication and lack of integration of the different disciplines. The misalignment of the systems team working in an iterative-v lifecycle while the development team were using Agile SCRUM meant that a different approach was needed. The project is a largely software based project for a large client, but the issues described above are seen in other projects with lesser and greater complexities. Ultimately the aim was to achieve co-engineering of the delivery, ownership of the work and reduce rework. Approach The focus of the approach was to apply the principles of the [Agile Manifesto] to Systems Engineering and to progress to cross-functional SCRUM teams. This meant trying to focus on self-organising teams, not hierarchical enforcement of methods of working, however guidance and advice was provided to the teams by managers and Agile specialists in the business. SCRUM was chosen as it was already in use in the company and the project for software, and was deemed suitable for adoption by the project for systems engineering as the functionality could be broken down into small enough elements to fit a 2 week lifecycle and the delivery cycle (3 months) also supported this way of working. Engagement of the customers and end users also helps with adoption; the customer in this case had started to get familiar with this way of working, but would also have benefitted by being taken on the journey with the team. Prior to the engagement of the teams to this approach, an exercise was undertaken to understand the mapping of the company processes into an Agile Systems Engineering approach. This meant that there was a framework for the introduction of Agile covering the usual artefacts that systems engineering would typically produce, including requirement specifications, compliance, and test cases and scripts. Consideration was also given to the SCRUM roles and how they would map to organisational roles.
The approach taken was first to introduce the concepts of Agile to the Systems Engineering team, through training sessions on Agile SCRUM, the process used, the ceremonies, and the roles in Agile. The figure below shows the SCRUM lifecycle for reference, but this is not described further here. There is recommended further reading in [Essential Scrum] for the established practices in SCRUM. [ScrumSense: Do Better Scrum] The training and introduction of concepts ensured that the team understood the theory and best practices in how the process works, including giving a better understanding of how the software team were working. The software team were working in 2 week sprints with customer releases approximately every 3 months (4 per year). The team structure prior to this initiative is shown below. There were two existing SCRUM software teams and a non-agile systems engineering team. The Product Owner was the Engineering Manager/Design Authority for the project. This arrangement saw that the Product Owner already had visibility of the systems and software engineering work. The two software SCRUM teams were focussed on delivering features for the programme across several different but related projects (based on a common core).
The next phase of the approach was a roll out of the process. The whole team (systems & software) were briefed on aligning the systems team to moving to SCRUM and the reasons behind this (see issues described earlier). Initially this was met with resistance by various members (as is typical with introducing any business change). The team were given the remit to determine what the appropriate implementation would look like for them (self-organising, empowerment). Reservations were held in particular by some software team members where SCRUM was seen as a software methodology and that introducing systems engineering would adversely affect their velocity, productivity and statistics, and hence the team decided that systems engineering should run their own SCRUM team to gain familiarity with the process. The team decided that the systems sprint (2 weeks duration) would be offset by 1 week to the software sprint to get over the chicken and egg situation where developers are waiting for specifications and testers are waiting for a build to test. This meant that when planning for the next sprint each team already knew what had been committed to by the other team(s) and could then plan development and testing accordingly. The new team structure is shown below. This now consisted of 3 SCRUM teams 2 software and 1 systems. The Product Owner is now closer to the teams and the Software Manager moves to more of a supporting role to the team and looks to help balance the resource between the 2 teams.
The structure then necessitated a SCRUM of SCRUMs to co-ordinate and address issues across the programme. This was colloquially referred to as the Management SCRUM. This approach was adopted in common with best practice in Agile SCRUM [Essential Scrum]. The purpose of this SCRUM of SCRUMs was to highlight any issues across the teams and was focussed on the following questions: What did the team do yesterday? What is the team doing today? Is any support required from management or elsewhere? (This includes any specific resource requirements) Are there any dependencies the team are waiting for? The SCRUM of SCRUMs is made up of the scrum masters from each team, the product owner, test manager and other engineering managers. The scope of the SCRUM of SCRUMs grew to include project management as wider engagement spread to relate the work being progressed into the schedule and reporting to the client. The programme schedule was adapted to the sprint structure and updated after sprint planning to show what was intended to be delivered in each sprint. The systems SCRUM team were now getting to grips with the process and methodology. The decomposition of traditional tasks to stories that could be completed in a 2 week sprint initially met with different approaches of how to get these to manageable chunks. Different members of the team took different approaches, with some ending up with 2 week tasks, and others decomposing to minute elements. After a few sprints the team consolidated their approach to get to 1-3 day tasks. This would include items such as regression test cycles (2 days), requirements specifications broken down to functional parts (e.g. tasks for UI requirements, back-end processing requirements and alarm requirements), and test reports. This meant that the development teams could be fed features that could be implemented without waiting for complete specifications. An important item to note is that as an existing project an architectural framework already existed and therefore parts of the design could be implemented into that overall framework that would still fit the
overall system design. For taking this approach on new projects it is imperative that this is present else there is a risk that what is implemented doesn t fit the end system design. As is typical with any new SCRUM team, several sprints were required to establish the team s velocity and what they could realistically commit to in the sprint. Other issues encountered were when dependencies from other teams or the client did not arrive when promised, meaning that an element of mid-sprint replanning would take place. The arrangement of 2 software and 1 system SCRUM teams continued for around 6 sprints; the systems team had now become much slicker at the process planning and refinement would be progressed with ease. This triggered the move to combine the teams to true cross-functional teams that could be focussed on feature delivery (design, implement test) for a work stream. This transition was enabled by the software team having a 3 week sprint to align the two cycles. The roles of the Software Manager and Test Manager are to support the teams and co-ordinate activities between SCRUM teams (for example scheduling availability of test systems for each feature, and release co-ordination). This end team structure is shown below. The programme manager also supports all three teams with planning and client liaison. This updated structure has now been in place and is delivering benefits outlined below. The teams are engaged and much improved communication has been seen. Dependencies and responsibility for the feature stream is now within the team. Daily stand-ups ensure that the team know what they are all doing and what issues they have and they can then be resolved by the team.
Results The result of this process and its application are outlined below: Greater alignment of design to code to test execution Reduction in defects Delivery stream focussed teams Better communication and integration of the teams Common metrics across teams Process alignment Demonstration that systems engineering can run in SCRUM Empowerment of teams Engagement of project management and scheduling in SCRUM Responsibilities as a SCRUM team rather than individual discipline teams. In terms of addressing the project issues, the results have shown that barriers have been broken down between the disciplines, leading to greater understanding of requirements early on by developers such that compliant solutions are developed together. The closer integration of developers to design and test to developers has reduced or captured and rectified any defects early on. Through the daily stand-ups and ceremonies the team know what the status is and when builds will be available to test. The sprint retrospectives also provide a constant feedback and improvement loop for the team to refine the process. The revised organisation also made the Product Owner have a more direct relationship with the SCRUM teams. Conclusions The process of bringing a systems engineering team through the journey to SCRUM working is achievable in a practical way, and the principles can work for systems engineering and engender co-engineering. The practical application of SCRUM to systems engineering requires the team to understand the principles and reasons behind moving to such an approach. It was found to be important that the engagement of the team in the process and adoption is important in a successful application. Efficiencies gained around process alignment between software engineering and systems engineering can be found, along with reductions in design misalignment. An effect of this process is now the desire to have one engineering plan to cover all activities rather than split on the traditional systems/software disciplines. The benefits outlined above were achieved within normal working on the project, training of staff was done by internal resources and embedded within the project team; project milestones still needed to be met during the transition phases. Throughout the process, the focus on delivering value for the project was embedded (this includes customer deliverables, as well as system functionality certain documentation is still very important to the customer, and some relate to payment milestones). The principles explored herein could readily be applied to larger projects with greater complexity. Consideration in this case should be given to principles for Agile at scale (including SAFe and LeSS). It should be noted that the introduction of SCRUM has helped in a number of ways outlined above, but it is not a solution to all problems. Project issues will still exist, especially with clients who are not familiar with Agile working. SCRUM is not necessarily the most appropriate method of working for all projects, and the project teams should be empowered to work in the methodology that gives most value and efficiency to the end product that is to be delivered. The project team involved here are now converts to this way of working and are championing this on other projects within the company.
References Tag [Agile Manifesto] [Essential Scrum] [ScrumSense: Do Better Scrum] Reference Details www.agilemanifesto.org Essential Scrum: A Practical Guide to the Most Popular Agile Process, Kenneth S. Rubin, 2012, ISBN 0-13-704329-5 http://www.scrumsense.com/wpcontent/uploads/2014/03/dobetterscrum-v3.02.pdf