Formal Methods in Resilient Systems Design using a Flexible Contract Approach

Transcription

1 Formal Methods in Resilient Systems Design using a Flexible Contract Approach Sponsor: DASD(SE) By Dr. Azad M. Madni 9 th Annual SERC Sponsor Research Review November 8, 2017 FHI 360 CONFERENCE CENTER 1825 Connecticut Avenue NW, 8th Floor Washington, DC SSRR 2017 November 8, 2017

2 Outline Project Snapshot The Team Use Cases Resilience Contract Formal Modeling Approach Multi-UAV Control Application Expected Results Transition SSRR 2017 November 8,

3 Project Snapshot System resilience a key requirement for military systems growing system complexity and a variety of disruptions need for long-lived systems capable of adapting to expected, unexpected, and unanticipated dynamic operational demands Current approaches ad hoc, piece-meal, do not scale difficult to verify model correctness and validate behaviors difficult to assess their long-term impact Environment uncertain, partially observable noisy sensors, equipment failures, communication loss, lack of visibility Challenges current methods inadequate for V&V of such systems Approach resilience contract (probabilistic + formal); CBD + POMDP Application multi-uav SoS management and control Expected Result resilience contract-driven multi-uav control prototype Transition defense organizations and aerospace companies SSRR 2017 November 8,

4 Our Team SSRR 2017 November 8,

5 Use Cases Too numerous! - multiple variations in state combinations Good news - fall into discernible patterns when abstracted their deployment / operation follows patterns e.g., deployment, enroute, action on objectives, redeployment details of each pattern differ for each scenario instance Factors influencing pattern differences METT-TC: Mission, Enemy, Terrain and weather, Troops and support available, Time available, and Civil Considerations SSRR 2017 November 8,

6 Resilience Contract Hybrid modeling construct with learning capability for stochastic/probabilistic systems partial observability, noisy sensors, uncertain environment key trade-off: degree of formality (V&V) vs level of flexibility (resilience) developed at design time, trained during operational use ( learning ) Comprises: traditional contract flexibility: relax assume-guarantee assertions in traditional contract Partially Observable Markov Decision Process (uncertainty handling) in-use learning (hidden states, transitions, emissions) pattern recognition Applications resilient multi-uav swarm control resilient autonomous vehicle SoS network dynamic mission assurance SSRR 2017 Copyright November 8, 2017 Azad M. Madni 6

7 Resilience Contract (RC) Policy Execution RC evaluates POMDP reward; typical responses: Keep going Stop Enforce trajectory to a safe state Notify support team SSRR 2017 November 8,

8 Engineered resilience is messy incompatible with invariant methods requirements can be imprecise (especially initially) actions can be unclear (especially initially) system states can be ambiguous (partial observability) Want a formal methodology consistent with theorem-proving key tradeoff: flexibility (messy problem) vs. formality (V&V) Why RC? Approach: probabilistic + formal modeling relax assumption-guarantees in traditional contract enables dealing with messiness while being compatible with formal V&V employ POMDP accounts for invariant knowledge, makes provision for inuse learned knowledge functions like a closed-loop control system results of actions are observed and used to determine next actions SSRR 2017 Copyright November 8, 2017 Azad M. Madni 8

9 Partially Observable Markov Decision Process (POMDP) Rationale many real world problem situations not fully observable Markov assumption invariably holds Defined by: set of: states S, actions A, observations O transition model, reward model, observation model Markov assumption applies to transition model optimal policy depends only on current state Partially observable environment current state not necessarily known agent cannot execute optimal policy for that state SSRR 2017 November 8,

10 Incorporating Flexibility in POMDP Relax time-invariance restriction (state space, action space) Add evaluation metric to determine best action Update emission / transition probabilities of hidden states Add concept of time SSRR 2017 November 8,

11 Incorporating Flexibility in POMDP SSRR 2017 November 8,

12 Key Insight Assert-guarantee construct in CBD is associated with fixed assertions For flexibility, we replace assert-guarantee with belief-reward (or penalty) construct requisite flexibility for resilience mostly preserves model verification and test benefits of CBD SSRR 2017 November 8,

13 State Transition Model SSRR 2017 November 8,

14 State Transitions: Details Some transitions are labeled with belief values e.g., b (failed) >= 0.95 is threshold of transition from normal motors to failed motor o i.e. transition happens if belief >= 0.95 that a motor has failed Some transitions have fixed assertions e.g., failedmotor and Operational Transition from Evaluate Environment to Auto Plan Enabled has three beliefs with different probabilities in our example Auto Planner determines the course of action to take based on environment beliefs, motor condition beliefs, and goals action taken: one that maximizes reward or minimizes penalty SSRR 2017 November 8,

15 Heterogeneous UAV Swarm Control: Example SSRR 2017 November 8,

16 MDP Model: UAV Flight Example 1 obstacle In this example, a UAV navigates from the start position to the goal by observing and avoiding obstacles. SSRR 2017 November 8,

17 MDP Model: UAV Flight Example 2 obstacle In this example, a UAV detects a critical fault condition and lands SSRR 2017 November 8,

18 Layered Architecture User Interface -Initial Conditions -Disruption Injects -Visualization Probabilistic Behavior Model -POMDP Selective Fidelity Physics Model - x ሶ = f x + g x u Data Sources -Sensors -Vehicle -Environment SSRR 2017 November 8,

19 UAV Modeling Just enough fidelity capture nonlinear dynamics of quadcopter takes into account aerodynamic drag coefficients for translational motion easily replicated (for SoS) waypoint navigation and following specified trajectory basic sensor model basic collision avoidance algorithms Requisite flexibility introduce and test-drive resilience concepts SSRR 2017 November 8,

20 Exemplar Swarm Control Architecture SSRR 2017 November 8,

21 Exemplar Inflexible Contracts Two inflexible contracts at the next instant, if obstacle ahead, then turn left at the next instant, if no obstacle ahead, then continue path SSRR 2017 November 8,

22 Extending Inflexible Contracts Can extend previous inflexible contracts to include threats threats can be on the left, right, or unsure define concept of belief state and penalty function evaluate belief state based on sensor inputs, and then optimize penalty function For example, a simple function might look like this: P(leftThreat)*10 + P(rightThreat)*(-100) + (1- (P(rightThreat)+P(leftThreat))*(-1))) If P(leftThreat) is 0.95, the P(rightThreat) is.01, and the probability that we can't decide is 0.04 then we get a penalty of = 8.49 if we turn left If on the other hand we decide to turn right, the penalty function becomes.01(10) +.95(-100) +.04(-1) = So, we would want to turn right because the penalty for turning left is positive, but the penalty for turning right is negative (i.e., reward) SSRR 2017 November 8,

23 Modeling Construct: Sense, (Re) Plan, Act (Re) Plan Executable Option Act Belief State Impact Sense Observe Environment Observe SSRR 2017 November 8,

24 Update Iterations S1 (threat on left) b0 Belief Vector Initial Belief S2 (threat on right) Action = observe o: threat on left b1 b0 Belief Vector b 1 s i = P(o s i, a) σ sj S P(s i s j, a)b 0 (s j ) P(o a, b)) SSRR 2017 November 8,

25 Technology Platform: Workflow Custom Java Bridge AnyLogic Java Code Python Code SSRR 2017 November 8,

26 Markov Decision Process Model for Multi-UAV SoS SSRR 2017 November 8,

27 Partially Observable Markov Decision Process Model for Multi-UAV SoS SSRR 2017 November 8,

28 Physics Model Hierarchy Quadcopter Sense Plan Act Sensor Packages Waypoint Navigation Planner Position Controller Attitude Controller Physics Model Obstacle Detection Set Waypoints Ordinary Differential Equations Path Generation Waypoint Database SSRR 2017 November 8,

29 Physics Model s Assumptions Quadrotor is a rigid body with symmetric mass distribution Propellers are rigid Center of gravity and body fixed frame origin are co-located Earth s gravitational field (g), quadrotor s mass (m) and body inertia matrix (J) are constants Thrust factor( kn) and torque factor(km) of motors are constants Inertia of motors and rotors are negligible Aerodynamic drag force is proportional with translational velocity Body angular velocities [p, q, r] in body frame and rate of change of Euler angles [ϕ,θ,ψ ] in earth frame are equal if perturbations from hover condition are small vehicle doesn t make very aggressive maneuvers o important when developing collision avoidance algorithms easier to determine the position using a parametric curve without running full dynamics model o error in position is very small (< 0.01m or 1 cm) SSRR 2017 November 8,

30 Parametric Curve to Determine Position Waypoint-to-waypoint navigation is modeled using Waymore s Standard Scoring Function Wymore s function is: Score=1/(1+(((B-L)/(V-L))^(2*S*(B+V-2*L)))) In our case, this function takes the following form: Where: t is simulation time Position =1/(1+(((B-L)/(t-L))^(2*S*(B+t-2*L)))); B is midpoint time between two waypoint S is minimum speed (S=1/(B-L)) L and U are required time from a waypoint to keep X,Y, Z bounded SSRR 2017 November 8,

31 Position Curve For a quadcopter to go a straight 10m line with 45 degree slope Maximum Slope (speed) L B U SSRR 2017 November 8,

32 Obstacle Avoidance: Deterministic Appraoch Initial Set of Waypoints: Time (s) X (m) Y (m) Z (m) Final Set of Waypoints: Short stop to determine next course of action New waypoint Time (s) X (m) Y (m) Z (m) Gets to final waypoint with a delay SSRR 2017 November 8,

33 Transition Have MOUs with Three Aerospace Companies Have Two Government Agencies as Transition Sites Have given periodic demos as we continue to make progress Have identified unique constraints of potential transition sites and their legacy investments they wish to preserve Have taken that into account in specifying our technology platform Intend to transition incrementally to continue to get customer feedback Publish with customers at our transition sites (e.g., CSER) SSRR 2017 November 8,

34 Hardware Testbed Plan: Piggyback high-level trajectory computer Quadcopter motors driven by Arduinobased flight controller (MultiWii) Full IMU: 3-axis accelerometers, rate gyros, magnetometer Ability to incorporate external GPS sensor Inputs for control values (throttle, rollpitch-yaw) Raspberry Pi with camera, OpenCV for image processing Position, attitude computed from IMU and pose estimation from camera image Synthesize control inputs from POMDP-driven trajectory calculation SSRR 2017 November 8,

35 Lab Setup SSRR 2017 November 8,

36 Where We Are Headed Model Enhancements o Extend POMDP - assign confidence levels to state estimates o Incorporate state additions - actions in POMDP o Incorporate multi-state, multi-observation POMDP-based RCs Integration and User Hardening o Incorporate means for integrating UAV actions into SoS actions o User harden software for use by others o Document usage scenarios for prospective users Transition o Develop detailed transition plans (industry, government) o Continue to follow built-in demonstration and transition points o Continue to demonstrate to transition partners to maintain their buy-in SSRR 2017 November 8,

37 References Madni, A.M., Sievers, M., A Flexible Contract-Based Design Framework for Evaluating System Resilience Approaches and Mechanisms, ISERC 2015, Nashville, Tennessee. Sievers, M., and Madni, A.M., "A flexible contracts approach to system resiliency." Systems, Man and Cybernetics (SMC), 2014 IEEE International Conference on. IEEE, Madni, A.M., Jackson, S., "Towards a conceptual framework for resilience engineering." Systems Journal, IEEE 3.2 (2009): Goerger, S.R., Madni, A.M., and Eslinger, O.J., "Engineered resilient systems: a DoD perspective." Procedia Computer Science 28 (2014): Madni, A.M. and Sievers, M. Model Based Systems Engineering: Motivation, Current Status and Needed Advances, accepted for publication in Systems Engineering, 2016 Neches, R., and Madni, A.M. "Towards affordably adaptable and effective systems." Systems Engineering 16.2 (2013): SSRR 2017 November 8,

38 Thank You! SSRR 2017 November 8,