Activity (AMSAA) Resources

Transcription

1 Army Materiel Systems Analysis Activity (AMSAA) AMSAA Dr. David Mortin 21 March 2018 DISTRIBUTION STATEMENT A - Approved for public release. Distribution is unlimited 1

2 Design Modeling Assessment tools Scorecard Reliability Growth Models Data Sample Data Collection CBM Metrics Analysis 2

3 Preparing for a Challenging Environment 3

4 Reliability Scorecard Provides standard method to identify reliability risk areas Based on IEEE, academia, Raytheon, Alion, RIAC, GEIA recommendations, best practices, etc. Structured approach allows refinement and improvement. General Scorecard has 8 Categories (40 elements) Software systems version has 7 Categories (57 elements) Sample Elements Reliability Requirements and Planning Training and Development Reliability Analysis Reliability Testing Reliability activities that are integral to design and testing are clearly identified and incorporated into the program Integrated Master Schedule Sufficiently-sized reliability engineering staff is directly tied to the design team Conduct failure modes, effects, and criticality analysis (FMECA) and fault tree analysis (FTA); crosswalk to low-level testing and a failure mechanism analysis to ensure programmatic coverage Conduct low-level testing early to identify and mitigate failure modes. Conduct highly accelerated life testing (HALT) and highly accelerated stress screening (HASS) Supply Chain Management Failure Tracking and Reporting Validation and Verification Reliability Improvements Plan and establish mature and welldocumented manufacturing procedures Utilize failure reporting, analysis and corrective action system (FRACAS) Modify testing and procedures based on failure occurrences Document lessons learned 4

5 Reliability Design Analyses Electronics in an Army Hand Held Device Initial Design Design after Physics-of- Failure Vibrations Analysis Thermal Analysis Provided Additional Insights This Circuit Board Fails Due to Vibration This Circuit Board Does Not (just by adding two screws) Design modification costing almost nothing changes product from being unreliable to reliable 5

6 System-Level Modeling Mitigate sources of failure Quickly develop solutions for failures Positively impact design Full-system Modeling provides tremendous insights that reduce testing and eliminate failures (pre and post fielding) 6

7 Root-Cause Analysis Identify weak spots and redesign in virtual environment not test-fixtest (Too Expensive!) Critical Components: Stresses, forces, accelerations, etc. are available for components in the model Models allow for quick visualization of multiple fixes or design enhancements 7

8 Sample Data Collection & Analysis Reliability Metrics Joint Base Lewis-McChord Ft. Carson Ft. Irwin Schofield Barracks Vilseck Poland Ft. Riley Ft. Campbell Ft. Stewart Ft. Rucker ANAD Ft. Bliss Ft. Hood Maintenance Hours Trends Kuwait New Maintenance Pilots Fixing Specific Field Issues 197 contractors at 11 CONUS and 3 OCONUS locations Over 7,044 ground systems and 508 aviation systems 8

9 CBM+: Fusing Data to Provide Actionable Information Digital Source Collector (DSC) & JPRO AMSAA has 2,300+ DSCs installed on vehicles at 8 world-wide locations Download to Laptop (Includes at platform Diagnostic Trouble Codes) AMSAA SDC&A Complete & Accurate Field Level Maintenance Data Server Analysis Algorithms Web-based CBM+ Tools Motor Pool Maintenance Management Fleet Management Vehicle Health Alerts Using Data Analytics of 200+ Billion Sensor Readings & Maintenance Events to Provide Actionable Information at Field and Fleet Levels 9

10 Typical CBM+ Data Collected Brakes Parameters Front Axle Speed Relative Speed, Front Axle, Left Wheel Relative Speed, Front Axle, Right Wheel Vehicle Speed Brake Switch CBM+ to Enable Data Driven Fleet Management Engine Parameters Accelerator Pedal Position Barometric Pressure Boost Pressure Coolant Level Engine Coolant Temperature Engine Load Engine Oil Pressure Engine Oil Temperature Engine Speed Fuel Rate Fuel Temperature Injection Control Pressure Intake Manifold Temperature Engine Percent Torque Vehicle Speed PTO Mode Diagnostic Trouble Codes Engine Transmission Brakes Transmission Parameters Transmission Oil Temp Transmission Range Attained Transmission Range Selected Transmission Output Shaft Speed Input Shaft Speed Transmission Actual Gear Ratio Transmission Current Gear Transmission Selected Gear Torque Converter Lockup Engaged Improved equipment Readiness Cost savings Reduced repair time Value added maintenance activities Correct parts ordered 10

11 Better Options for Reliability Prediction System 1 System 2 System 3 System 4 System 5 System 6 System 7 System 8 System 9 System 10 Demonstrated Prediction 1) Hard to keep handbooks up-to-date with current technology 2) Represent only a small portion of the overall system failure rate 3) Does not consider critical factors that influence reliability (e.g. thermal cycling) System 11 System 12 System 13 System Mean Time Between Failure (Hours) 4) Does not provide insight on how or why the system will fail 5) Data not current (lagging years) 11

12 Non-Mission Capable Actions (per XXXX miles) Economic Useful Life Analysis Based on 6 years of Sample Data Collection field data Vehicles range in age from 3 to 12 years old Represents the 10 th, 50 th, and 90 th percentiles for each mileage bin Odometer Reading (miles) For this vehicle, no statistical indications of aging over the period Multiple ways for PSMs to analyze to ensure optimum Economic Useful Life decisions 12

13 Leveraging Data and Information to the Maximum Extent Possible Probability of Failure σ f Subsystem Strength Using Physics-Based- Reliability methodology to determine impacts of weight or mission changes for ground systems Field data from existing systems combined with engineering modeling 13

14 Reliability Metric Differences Pre and Post Fielding Environment Incident Classification Metric Test Field Test incidents classified using FD/SC (i.e. EMA, UMA, OMF, SA, EFF) Unscheduled maintenance actions classified using TM-10 PMCS (i.e. NMC) MTBSA, MTBEFF MTBNMCA, MTBUMA How can we compare test metrics to field metrics? 14

15 A Case Study Ground Vehicle Available Data Test Failure data collected via TIRs Detailed description of maintenance performed Field Unscheduled maintenance and associated part replacements data collected in field Basic fault description, correction narrative and failure description (if applicable or known) Mapped NSN s that cause NMC events in the field to test incident reports Common rule sets in the field were applied to test data 15

16 Ground System Reliability Metrics Adjustments to test data allow for a more direct comparison of test and field reliability metrics. Statistical tests can be utilized to determine if there are any significant differences in the test and field data sets. Reliability Comparison Mean Miles Between NMC Action (MMBNMCA) = Total Mileage # NMC Actions *Also contrasted to baseline requirement Another way to compare pre and post fielding reliability is to score the field data as was done in testing 16

17 Mean Time Between Failure Reliability Growth and its Impact On Support Costs 300 Idealized Curve Customer Test Initial DT LUT LUT Excursion IOT $894 M $1,103 M $869 M $1,701 M $2,457 M 114 CAP1 80 CAP2 175 CAP3 215 CAP4 220 Reliability Growth Planning, Tracking, & Projection tools along with the reliability scorecards are available for free to US Government organizations and supporting contractors 50 0 CAP Corrective Action Period Test Time (Hours) 17

18 Establishing Solid Reliability Growth Strategies Category Low Risk Medium Risk High Risk MTBF Goal (DT) MTBF Growth Potential < 70% 70-80% > 80% IOT&E Producer s Risk 20% % > 30% IOT&E Consumer s Risk 20% % > 30% Management Strategy < 90% 90-96% > 96% Fix Effectiveness Factor 70% % > 80% MTBF Goal (DT) MTBF Initial Time to Incorporate and Validate Fixes in IOT&E Units Prior to Test < > 3 Adequate time and resources to have fixes implemented & verified with testing or strong engineering analysis Time and resources for almost all fixes to be implemented & most verified w/ testing or strong engineering analysis Many fixes not in place by IOT&E and limited fix verification Risk assessment of curves now included in some reliability growth modeling tools 18

19 Stocks, Forecasting, and Requirements Assessment Billions Supply Chain & Inventory Analysis Selected Essential Stocks for Availability Methodology (SESAME) Optimum stock levels and identification of excess inventory $10 $9 $8 $7 $6 $5 $4 $3 $2 $1 $- 0-2 Years Insurance Items BDRO Safety Level LeadTime Rqmt 24 Months Dmd Econ/LOT Retn LOT-Excess Excess 2-5 Years Years Years Years 50+ Years NFD Demand Planning Analyzing accuracy of forecasts Impact of multi-level additive manufacturing and its potential effects on demand planning Army Prepositioned Stock Computations Optimum Stock Requirements Analysis Program Optimized packages for Class I, II, IIIP, IV and IX Analysis of Requirements, New Logistics Concepts, and PBL Arrangements Establish business based performance metrics to track efficiency and value added Assess requirements feasibility along with mission impact 19

20 SA and Contact Info Field Operations Sample Data Collection & Analysis Operational Sustainment Condition Based Maintenance Reliability Centered Maintenance 100% 95% 90% 90% SA 95% SA 99% SA 85% 85% SA 80% 75% $0.0 $0.5 $1.0 $1.5 $2.0 $ Safety Level (Billions) Reliability Center for Reliability Growth Physics of Failure Requirements Determination Test Support & Efficiencies Materiel Management Supply Chain Inventory Optimization Demand Planning Army Prepositioned Stocks Operational Energy Fuel Consumption Power Sufficiency Expeditionary Basing Energy Operational Logistics Planning Acquisition Support Logistics Footprint Level of Repair Life Cycle Cost Schedule Risk David Mortin, Ph.D david.e.mortin.civ@mail.mil 20