Architecture Based Analysis of System ility Synergies and Conflicts

Transcription

1 Architecture Based Analysis of System ility Synergies and Conflicts Barry Boehm, Jo Ann Lane, USC Kevin Sullivan, U. Virginia NDIA Systems Engineering Conference October 30, Outline Critical nature of the ilities Major source of project overruns, failures Significant source of stakeholder value conflicts Poorly defined, understood Underemphasized in project management Challenges for cyber physical human systems SERC Foundations efforts AFIT, GaTech, MIT, NPS, PennState, USC, Uva, WSU Stakeholder value based, means ends hierarchy Formal analysis of ility definitions and relations Architecture strategy synergies and conflicts

2 Importance of ility Tradeoffs Major source of DoD system overruns System ilities have systemwide impact System elements generally just have local impact ilities often exhibit asymptotic behavior Watch out for the knee of the curve Best architecture is a discontinuous function of ility level Build it quickly, tune or fix it later highly risky Large system example below Role-Based Ilities Value Diversity Bank of America Master Net; DoD?

3 Example of Current Practice The system shall have a Mean Time Between Failures of 10,000 hours What is a failure? 10,000 hours on liveness But several dropped or garbled messages per hour? What is the operational context? Base operations? Field operations? Conflict operations? Most management practices focused on functions Requirements, design reviews; traceability matrices; work breakdown structures; data item descriptions; earned value management What are the effects on other ilities? Cost, schedule, performance, maintainability? USC: COCOMO II-Based Tradeoff Analysis Better, Cheaper, Faster: Pick Any Two? Cost ($M) 9 (RELY, MTBF (hours)) (VL, 1) (L, 10) (N, 300) (H, 10K) (VH, 300K) For 100-KSLOC set of features Can pick all three with 77-KSLOC set of features -- Cost/Schedule/RELY: pick any two points Development Time (Months)

4 Outline Critical nature of the ilities Major source of project overruns, failures Significant source of stakeholder value conflicts Poorly defined, understood Underemphasized in project management Challenges for cyber physical human systems SERC Foundations efforts AFIT, GaTech, MIT, NPS, PennState, USC, Uva, WSU Stakeholder value based, means ends hierarchy Formal analysis of ility definitions and relations Architecture strategy synergies and conflicts Importance of Cyber Physical Systems Major gap in tradespace analysis capabilities Current ERS, DARPA tradespace research focused on physical system tradeoffs Range, payload, size, weight, lethality, power and fuel consumption, communications bandwidth, etc. Some focus on physical modularity, composability Current cyber tradespace research focused on software, computing, human factors tradeoffs security, safety, interoperability, usability, flexibility, adaptability, dependability, response time, throughput, etc. Gaps in capabilities for co design of hardware, software, and human factors; integration of tradespace analyses

5 GaTech FACT Tradespace Tool Configure vehicles from the bottom up Quickly assess impacts on performance Being used by Marine Corps Prioritized JCIDS ilities User View by Combatant Commands: Top priority first Intelligence, Surveillance, and Reconnaissance Comprehensive Persistent Survivable Integrated Timely Credible Adaptable Innovative Command and Control (note emphasis on Usability aspects) Interoperability Understanding Timeliness Accessibility Simplicity Completeness Agility Accuracy Relevance Robustness Operational Trust Logistics: Supply Responsiveness Sustainability Flexibility Survivability Attainability Economy Simplicity Logistics: Mit Maintenance Sustainability Responsiveness Attainability Flexibility Economy Survivability Simplicity Net Centric: Information Transport Accessible Capacity Accurate Timely Throughput Expeditionary Latency

6 Outline Critical nature of the ilities Major source of project overruns, failures Significant source of stakeholder value conflicts Poorly defined, understood Underemphasized in project management Challenges for cyber physical human systems SERC Foundations efforts AFIT, GaTech, MIT, NPS, PennState, USC, Uva, WSU Stakeholder value based, means ends hierarchy Formal analysis of ility definitions and relations Architecture strategy synergies and conflicts SERC Value Based ilities Hierarchy Based on ISO/IEC 9126, 25030; JCIDS; previous SERC research Individual ilities Mission Effectiveness: Speed, Physical Capability, Cyber Capability, Usability, Accuracy, Impact, Endurability, Maneuverability, Scalability, Versatility Resource Utilization: Cost, Duration, Personnel, Scarce Quantities (capacity, weight, energy, ); Manufacturability, Sustainability Protection: Security, Safety Robustness: Reliability, Availablilty, Maintainability, Survivability Flexibility: Modifiability, Tailorability, Adaptability Composability: Interoperability, Openness, Service Orientation Composite ilities tes Comprehensiveness/Suitability: all of the above Dependability: Mission Effectiveness, Protection, Robustness Resilience: Protection, Robustness, Flexibility Affordability: Mission Effectiveness, Resource Utilization

7 Affordability Improvements and Tradeoffs Means-Ends Framework: Affordability Get the Best from People Make Tasks More Efficient Eliminate Tasks Eliminate Scrap, Rework Simplify Products (KISS) Reuse Components Reduce Operations, Support Costs Value- and Architecture-Based Tradeoffs and Balancing Staffing, Incentivizing, Teambuilding Facilities, Support Services Kaizen (continuous improvement) Tools and Automation Work and Oversight Streamlining Collaboration Technology Lean and Agile Methods Task Automation Model-Based Product Generation Early Risk and Defect Elimination Evidence-Based Decision Gates Modularity Around Sources of Change Incremental, Evolutionary Development Value-Based, Agile Process Maturity Risk-Based Prototyping Value-Based Capability Prioritization Satisficing i vs. Optimizing i i Performance Domain Engineering and Architecture Composable Components,Services, COTS Legacy System Repurposing Automate Operations Elements Design for Maintainability, Evolvability Streamline Supply Chain Anticipate, Prepare for Change Architecture Strategy Synergy Conflict Matrix

8 Software Development Cost vs. Quality Relative Cost to Develop Very Low Low Nominal High COCOMO II RELY Rating Very High MTBF (hours) , ,000 Software Ownership Cost vs. Quality VL = 2.55 L = 1.52 Operational-defect cost at Nominal dependability = Software life cycle cost 1.26 Relative Cost to Develop, Maintain, Own and Operate Operational - defect cost = % Maint Very Low Low Nominal High COCOMO II RELY Rating Very High MTBF (hours) , ,000 8