HR001118S0022 COMPASS Proposers Day Questions and Answers

Transcription

1 HR001118S0022 COMPASS Proposers Day Questions and Answers Q1: What is the dividing line between TA1 and TA2? Is it based on time scale of goals and scope of actions? What are some examples of the sorts of intents and actions considered strategic vs. tactical? A1: The division between TA1 and TA2 is intentionally fuzzy. In general, tactical planning develops the playbook for implementing strategic goals at the operational level. TA1 focuses on the semi-steady state of the environment, where timelines are longer and the complexity is lower. TA2 addresses highly dynamic environments with potential instabilities and tipping points. See BAA Part II Section I.A.2 for an overview of scale and scope of TA1 and TA2 aims. Q2: Should TA1 and TA2 assume that TA2 will use outputs from TA1? Will there be an API defined? A2: Yes, interaction between TA1 and TA2 technologies is bidirectional. See BAA Part II Section I.A.3.c for discussion of TA1/TA2 interaction. Yes, per BAA Part II Section I.A.4.a, TA3 will provide APIs through which TA1 and TA2 performers can interact with the test range to both stimulate the environment and monitor effects. Q3: Is active probing mostly in TA2 or also TA1? How much probing is allowed? A3: Per the detailed descriptions of TA1 and TA2 in Part II Section 1.A.3.a. and Part II Section 1.A.3.b of the BAA, respectively, active probing is central to both TA1 and TA2. The number and extent of proposed probes will be decided during program execution and it will be considered as a metric of evaluation, as specified in section I.B. Q4: Can you clarify the distinction between and relationship between TA1 and TA2 probe actions? A4: TA1 aims at long-term assessment of intent and strategy. TA2 considers the associated short-term tactics and actions. As such, they will differ in temporal scale, specific actors, specific geographic locations, etc. That said, some probe actions may apply to both TA1 and TA2. See BAA Part II Section I.A.2, Part II Section 1.A.3.a., and Part II Section 1.A.3.b. Q5: Will the simulation specify a limited set of allowed probing actions, and how will the proposed probes be input into the simulation? A5: The simulation will seek to facilitate the probing actions proposed by TA1 and TA2 teams. The envisioned probes will be inserted into the simulation through the use of a hierarchical ontology. See BAA Part II Section I.A.3.d for further explication of the envisioned framework for integration of probes. Q6: Are the TA1 and TA2 solutions expected to be applicable across various domains (e.g., different locations, types of adversaries, degrees of hostility, time scale)? Or, should they focus on the specific domain simulation selected for the program? A6: Envisioned TA1 and TA2 approaches should be able to generalize across multiple, coevolving domains. This is addressed most explicitly in BAA Part II Section I.A.3. 1

2 Q7: Does TA3 integrate all TA1 and TA2 products into a single decision aid? Does it need to aggregate them into a unified view? A7: No, it is not expected that all TA1 and TA2 products will aggregate as a unified view. TA3 will incorporate some subset of TA1 and TA2 products, where this subset may evolve throughout the program. Part II Section I.A.3.d of the BAA described expectations for TA3 technologies. Q8: Will TA3 also allow the user to interact with the data from the T&E simulation? A8: Per BAA Part II Section I.A.3.d, TA3 will primarily support TA1 and TA2 teams interaction with the simulation. It will also provide the HMI that the users interact with the decision aids from TA1 and TA2. It is not intended for TA3 to develop views directly into the data, however, if such a capability is requested by the operational users, we might consider adding some simple utilities to examine the raw data. Q9: How will TA3 be evaluated? A9: See BAA Part II Section I.A.4 (Schedule/Milestones), BAA Part II Section I.B (Program Metrics), and BAA Part V Section A (Evaluation Criteria). Q10: Is it assumed that we are primarily dealing with a single state-level adversary? A10: No. In-scope approaches may start by considering simpler red/blue/white scenarios, and then generalize to multiple allies/adversaries. Q11: Is there an element of anomaly detection or attack recognition? A11: Yes, those technologies are in scope. Q12: Will the test range/simulation be available to performers for repeated in-house use, or only, e.g., during limited engagements? A12: Yes, the simulation test bed will be available to performers for in-house use. The simulation will be made available through a cloud-based infrastructure. Q13: How abstract or concrete will the domain simulation and scenario be? A13: This will depend on the approaches selected. Q14: Will all simulated/generated data be in English, or are performers expected to handle translation? A14: Per BAA Part II Section I.A.3.d, all data will be in English. Q15: Will models be provided, and can you give an example of what a model would look like? A15: No models will be provided. Per BAA Part II Section I.A.3, teams are encouraged to leverage existing modeling capabilities. 2

3 Q16: How prescriptive are the proposed game theory and control theory approaches? Will you consider alternatives? A16: Per BAA Part II Section I.A.3.a, proposed approaches are notional. Yes, alternatives are welcome. Q17: Will TA1 and TA2 be evaluated with a human in the loop, who makes choices and interacts with the T&E simulation based on TA1/TA2 recommendations? Or, are TA1 and TA2 evaluated based on their assessments of snapshots from the T&E simulation output? A17: Initially, TA1 and TA2 teams will be evaluated against the simulation. Later in the program, approaches will be evaluated with a human in the loop. See BAA Part II Section I.A.3.e for further information on Test and Evaluation. See BAA Part V Section A for Evaluation Criteria. Q18: Will TA1 and TA2 teams interface directly with the T&E team? A18: Yes. See BAA Part II Section I.A.3.e for further information on Test and Evaluation. Q19: What level of fidelity/complexity should be expected, e.g., motive/intent/actions, between technical areas? A19: Complexity of probes will not be explicitly specified. Teams are directed to consult the PMESII documents referenced in the BAA Part II Section I.A.1 for examples, and to argue why their proposed approach is sufficient. These documents include: Mazarr, Michael (2016) and Office of the Secretary of Defense (2009). In addition, the level of complexity will be driven by the nature of the scenario and will be defined in collaboration with the performers and the evaluator. Q20: In a universe of infinite potential actions, are there a specific subset of user-level decisions to reference/expect? A20: No specific subset of decisions will be prescribed. It is expected that proposals will demonstrate an approach to bound this problem. Q21: Will teams have access to a community of subject matter experts at the operations and end-user level? A21: The government (T&E) team will aim to provide this type of support and expertise. Q22: If TA1, TA2 and the T&E simulation end up using the same PMESII models, is there concern that it would be too easy to predict/recognize adversary intent? A22: Yes. Evaluators will work to ensure that reverse engineering the system is not possible. However, even if the technology providers and the simulation use the same models, the parameters of those models will vary, the environment will contain ambiguities and the subversive actions of the red actor included in the simulation will be hidden from the tool providers. Q23: Focusing on TA1, there is a host of international relations theory and security studies knowledge which provide helpful guideposts, and analytic and predictive power for anticipating behavior of an 3

4 adversary in a strategic setting. Is there room in the program for computational representation of relevant parts of these models? A23: Yes, per BAA Part II Section I.A.3, leveraging existing models or instantiating existing theory as models is encouraged. Q24: Will TA3 provide a common ontology and logical languages (e.g., triples), or is TA3 to map across all possibilities? A24: Per BAA Part II Section I.A.3.d, TA3 will work with the tool providers (TA1 and TA2) to define a common interface (ontology, or a data representation) that capture the characteristics of the data that will be used by the program. Q25: Any sense of what kinds of data will be emphasized? E.g., physical transport, power infrastructure, financial markets, military movements? A25: This will be up to each team and will depend on the proposed approach. Q26: Will the nature of the data sources be resolved at the start of the program? That is, will there be an API to the simulator from the beginning, or will it remain open-ended for much of Phase I? A26: As specified in the BAA (Table 2, p. 19), at program kickoff the government team will provide an initial set of data to assist technology providers in understanding the characteristics of the environment and to start the development of their systems. However, it is anticipated that additional datasets will be required to capture the complexity and nuanced conditions of gray zone engagement. A specific timeline of the additional data provisioning is not yet known, but it is anticipated to occur on a sixmonth schedule. It is also envisioned that individual teams will propose and leverage their own data to drive their technical approach, and to make that data available to all participants, if it benefits the overall program. Q27: Will data be ITAR or export controlled? A27: It may be. This will depend on the specific datasets considered. Q28: Could you elaborate on the relative importance of the program s envisioned benefits: reduce strategic surprise; win the grey zone struggle; avoid inadvertent escalation? A28: The program envisions supporting better understanding in each of those respects. Envisioned capabilities will provide options to a commander, who can then evaluate specific priorities. See BAA Part II Section I.A.2 for high-level program goals. Q29: Can multiple TAs be proposed in one proposal? A29: No. See BAA Part II Section III.D.2. Q30: If submitting to both TA1 and TA2, how many proposals should be submitted? A30: Two. See BAA Part II Section III.D.2. 4

5 Q31: If bidding on all three TAs, how many proposals should be submitted? A31: Three. See BAA Part II Section III.D.2. Q32: On page 7 the BAA states: COMPASS information products will feed the Course of Action (COA) planning phase during which blue commanders develop reactive and pro-active campaigns. Is COA development within scope? A32: No, COA development is out of scope. In fact, all the steps of the Joint Operations Planning Process (which includes mission analysis, COA development, analysis, comparison, and approval) are outside the scope of this program. The BAA will be amended to reflect this clarification. Q33: On page 9 (11, and 13) the BAA states: TA1 (2 and 3) proposers may address one or more of the following topics. Are there any assumptions regarding the composition of the teams and the extent of the proposed effort? A33: The composition of the teams and the extent of the effort will be determined solely by the contents of the proposals and the quality of the technical approach. The BAA will be amended to reflect this clarification. 5