Download Free The Role Of Simulation In Operational Test And Evaluation Book in PDF and EPUB Free Download. You can read online The Role Of Simulation In Operational Test And Evaluation and write the review.

The objective of this study is to formulate a set of guidelines for the application of computer simulations and models to the problem of assessing weapon system performance. Specifically, the study addresses the question: how the OT & E community can use models and simulations more effectively in evaluating weapon systems. The methodology for conducting operational testing is multidimensional; the number of variables precludes a hard set of rules for what, when, and how best to simulations. The primary audience is the decisionmaker in the Air Force and in the Office of the Secretary of Defense (OSD) who must support the development of more capable methods by which to formulate his program support decisions. This study focuses on the application of simulation techniques to the problem of assessing a weapon system's performance in its operational environment. Because of the exhaustive work that has been done in the area of simulation and modeling, the study avoids detailed discussions of simulations. Testing is a key element in the defense system acquisition process because it provides data for evaluating system development progress.
Simulation has an essential role in test and evaluation. It provides efficiencies in U.S. Army acquisition strategies of mission-based test and evaluation and integrated testing and training, as evident with the Boeing Engineering Development Simulator during the Apache Block III force development test and experimentation testing. Simulators provide a representative environment where testers can safely test a product's or system's mission effectiveness. Ideal for integrated testing, simulators combine developmental and operational testing as they enable safe simultaneous testing of multiple elements. Additional benefits of cost and risk reduction were realized as well as aviation doctrine development.
For every weapons system being developed, the U.S. Department of Defense (DOD) must make a critical decision: Should the system go forward to full-scale production? The answer to that question may involve not only tens of billions of dollars but also the nation's security and military capabilities. In the milestone process used by DOD to answer the basic acquisition question, one component near the end of the process is operational testing, to determine if a system meets the requirements for effectiveness and suitability in realistic battlefield settings. Problems discovered at this stage can cause significant production delays and can necessitate costly system redesign. This book examines the milestone process, as well as the DOD's entire approach to testing and evaluating defense systems. It brings to the topic of defense acquisition the application of scientific statistical principles and practices.
The Department of Defense has attempted to use recent advances in modeling and simulation to improve the acquisition process for weapons systems. This Simulation Based Acquisition brought advances in the process, but considerable disagreement remains over the universal applicability of this approach. This paper focuses on the challenges of applying modeling and simulation to the Test and Evaluation of a weapon system with significant Pilot-Vehicle interface concerns. The Standoff Land Attack Missile Expanded Response (SLAM ER) is an aircraftlaunched missile with GPS/INS guidance for navigation to the target area and Man In The Loop (MITL) control in the terminal phase. The MITL control is conducted through a two way video and control data link which transmits infrared video from the missile seeker to the control aircraft and guidance update commands from the pilot back to the missile. After initial fielding of the weapon system, two preplanned product improvement programs were begun to add both an Automatic Target Acquisition (ATA) functionality to aid in pilot target identification as well as a capability to engage moving targets at sea (ASuW). Both Software in the Loop and Hardware in the Loop simulations were available for the testing of both these SLAM ER improvements. This paper focuses on the utility of this simulation support in the Test and Evaluation prior to delivery to the operational users. Though the management issues of cost and schedule can be large drivers in the use of modeling and simulation, this paper will focus on the performance aspect of weapon system evaluation. Through the course of both the ATA and ASuW evaluations, simulation was able to provide very limited contributions to evaluations of system performance when MITL control was a concern. Simulation was useful in providing data on easily quantifiable parameters, such as seeker scan rates. However, flight tests with a physical prototype provided the only effective data when subjective measures such as pilot workload and pilot target identification were a concern. The simulators available did not effectively replicate the pilot interface or workload environment to the level required for valid MITL data. Only when an issue with the pilot interface was easily defined in quantifiable engineering data was simulation useful in identifying a possible solution--one that had to be further evaluated in subsequent flight testing. As the quality of models and simulations continue to improve with advances in computing, modeling of the pilot vehicle interfaces may improve in the future. Until that time, management controls will be essential to correct application of modeling and simulation in areas where MITL is a concern. The development of models and simulations should begin early in the acquisition effort with robust verification and validation devoted to the pilot interface. Early identification of the areas in which simulations can contribute to the MITL evaluation effort as well as recognition of the limitations of models and simulations. Finally, the validated simulations should be viewed as an enhancement to the evaluation effort with live testing of the physical prototype forming the basis of the MITL evaluation, particularly when the system approaches the final phases of Developmental Testing and prepares for Operational Testing.
The U.S. Army Test and Evaluation Command (ATEC) is responsible for the operational testing and evaluation of Army systems in development. ATEC requested that the National Research Council form the Panel on Operational Test Design and Evaluation of the Interim Armored Vehicle (Stryker). The charge to this panel was to explore three issues concerning the IOT plans for the Stryker/SBCT. First, the panel was asked to examine the measures selected to assess the performance and effectiveness of the Stryker/SBCT in comparison both to requirements and to the baseline system. Second, the panel was asked to review the test design for the Stryker/SBCT initial operational test to see whether it is consistent with best practices. Third, the panel was asked to identify the advantages and disadvantages of techniques for combining operational test data with data from other sources and types of use. In a previous report (appended to the current report) the panel presented findings, conclusions, and recommendations pertaining to the first two issues: measures of performance and effectiveness, and test design. In the current report, the panel discusses techniques for combining information.
This book describes the application of simulation techniques to the operational test and evaluation (OT&E) of command support systems (CSS). OT&E is essential for ensuring that systems are effective, suitable for service and meet the end users' needs. Simulation can be used in OT&E to simulate entities that cannot be obtained for real-world OT&E and to evaluate the effectiveness of systems in controlled environments. This book defines the relationships between command and control (C2), CSS, systems engineering, verification and validation, OT&E and modelling and simulation. This book describes how simulation techniques have been used to improve OT&E of CSS and how simulation models are verified, validated and accredited. This book describes how the effectiveness of an OT&E program can be measured in terms of its cost effectiveness with respect to gathering evidence to support the resolution of critical operational issues (COIs). This book describes a proposed methodology for applying simulation in OT&E of CSS to help gather data to resolve COIs when it is impractical to conduct sufficient real-world OT&E to gather sufficient data to resolve COIs.
The U.S. Army Test and Evaluation Command (ATEC) is responsible for the operational testing and evaluation of Army systems in development. ATEC requested that the National Research Council form the Panel on Operational Test Design and Evaluation of the Interim Armored Vehicle (Stryker) to explore three issues concerning the initial operation test plans for the Stryker/Interim Brigade Combat Team (IBCT). First, the panel was asked to examine the measures selected to assess the performance and effectiveness of the Stryker/IBCT in comparison both to requirements and to the baseline system. Second, the panel was asked to review the test design for the Stryker/IBCT initial operational test to see whether it is consistent with best practices. Third, the panel was asked to identify the advantages and disadvantages of techniques for combining operational test data with data from other sources and types of use. In this report the panel presents findings, conclusions, and recommendations pertaining to the first two issues: measures of performance and effectiveness, and test design. The panel intends to prepare a second report that discusses techniques for combining information.
The purpose of this thesis is to analyze the integration of Modeling and Simulation (M & S) into the U.S. Army Operational Test and Evaluation (OT & E) process. The elements. shortfalls, and recurring problems associated with the OT & E system are examined with a focus on those that can be addressed by M & S. Current and future M & S architectures are outlined to provide a base of understanding for the applicability to the OT & E process and issues. Analysis of the potential strengths and weaknesses of M & S in addressing OT & E problems and issues are presented. Lessons learned from past OT & E efforts are also analyzed for process improvement through M & S integration. From this analysis, a set of recommendations in the area of M & S integration into Army OT & E are formulated and offered.
The purpose of this thesis is to analyze the application of Modeling and Simulation (M&S) within the Army Operational Test and Evaluation (OT&E) process in support of weapon systems acquisition. This thesis considers the Army's current acquisition process, M&S technologies, infrastructure, and policies that guide the Program Manager (PM) in the application of modeling and simulation in operational testing. An analysis of the potential strengths and weaknesses of M&S in addressing OT&E issues is presented. Lessons learned from past OT&E efforts are analyzed for process improvement through M&S applications. The analysis indicates that M&S is a viable tool for assisting the PM in completing OT&E. M&S techniques can assist in test design validation, expand testing in areas of limited resources and environmental concerns, and validate live testing data. From this analysis, a set of recommendations are formulated, indicating where the PM can integrate M&S into the OT&E process.