Workshop on Research Challenges in Modeling & Simulation for Engineering Complex Systems


Computer-based models and simulations are vital technologies needed in advanced economies to guide the design of complex systems. Modeling and simulation is essential in areas such as the creation of smart cities and sustainable urban growth, aerospace, manufacturing, healthcare, security and defense, among others. However, the development and use of reliable computer models and simulations is today time consuming and expensive. Engineered systems are continually increasing in complexity and scale. Advances in modeling and simulation are essential to keep up with this growing complexity and to maximize the effectiveness of computer-based tools to engineer the increasingly complex systems that will be needed in the future. Further, recent technological advances such as cloud computing, novel computer architectures, big data, and the Internet of Things necessitate new developments in modeling and simulation to fully exploit these capabilities. This workshop will bring together leading researchers from disparate disciplines to identify and articulate the most important, impactful modeling and simulation research problems and questions that must be addressed to maximize the impact of new M&S technologies. The results of this workshop will inform future research investments in modeling and simulation, and ultimately, enable the creation of more effective and less costly engineered, complex systems.

The goal of this workshop is to define and articulate critical M&S research challenges in the design of engineered complex systems. The workshop will focus on five key areas: (1) conceptual modeling, e.g., to determine how teams of individuals from different disciplines can best create sophisticated, reliable models of complex systems, (2) advanced computational methods including topics such as exploitation of emerging computing capabilities and technologies for simulation, model checking and inference, (3) approaches to manage uncertainty and address model fidelity concerns, (4) approaches to enable and facilitate model reuse in order to accelerate and reduce the cost of creating effective computational models, and (5) determination of needs and assessment of the impact of advances in M&S in the aforementioned application domains.

The two-day workshop will take place on January 13-14, 2016 at the National Science Foundation in Arlington Virginia. Participation is by invitation only.

Click here for hotel information


Workshop Sponsors:

•   National Science Foundation

•   National Aeronautics and Space Administration

•   Air Force Office of Scientific Research

•   National Modeling & Simulation Coalition / National Training & Simulation Association

1.   Objective

The goal of this workshop is to identify and build consensus around critical research challenges in the modeling and simulation field – challenges whose solution will significantly impact and accelerate the solution of major problems facing society today. Although modeling and simulation has been an active area of study for some time, new developments such as the need to model systems of unprecedented scale and complexity, the well-documented deluge in data arising today, and revolutionary changes in underlying computing platforms are creating major new opportunities and challenges in the M&S field. For these reasons a workshop in M&S now is both timely and important both to the M&S research community as well as the numerous fields that depend on M&S technologies and techniques.

2.   Key Questions

For the purposes of this discussion, we informally define a model as a representation of an object or system for some purpose. A simulation captures salient aspects of the dynamic behavior of the modeled system over time. We note that modeling and simulation are closely related, but are distinct areas and each offers challenges in its own right. Implicit in both modeling and simulation are data that are used to quantitatively characterize important aspects of the system under investigation.  

The workshop will focus on five topic areas to generate challenges for modeling and simulation research:

1.   Conceptual modeling

2.   Computational methods: algorithms for simulation, model checking, and other types of inference

3.   Fidelity issues and uncertainty in modeling and simulation

4.   Model reuse, composition, and adaption

5.   Selected applications benefiting from advances in modeling and simulation

Each of these is briefly discussed next.

2.1   Conceptual Modeling

Although one of the first steps in the development of a model is the development of a conceptual model, such conceptual models have traditionally been informal, document-based. As the complexity of simulation models increases and the number of domain experts contributing to a single model grows, there is an increasing need to create formal, descriptive models of the system under investigation and its environment. This is particularly important for the engineering of complex systems where multiple system alternatives are explored, compared and gradually refined over time. The descriptive model of each system alternative — describing the system of interest, the environment and interactions between them — can serve as a conceptual model for a corresponding analysis or simulation model. Formal modeling of these descriptive, conceptual models poses significant research challenge:

•   How can models expressed by different experts in different modeling languages be combined in a consistent fashion?

•   What level of formality is suitable for efficient and effective communication?

•   What characteristics should a modeling environment have to support conceptual modeling in an organizational context — a distributed cognitive system?

•   What transformations of conceptual models to other representations are possible, and useful? What are the major impediments to realizing such transformations?

2.2   Computational Methods: Algorithms for Simulation, Model Checking and Other Types of Inference

The main reason for modeling is to extend human cognition. By expressing our knowledge in a mathematical formalism, the rules of mathematical inference implemented in computer algorithms can be used to draw systematic conclusion that are well beyond the natural cognitive ability of humans. For instance, simulation allows us to project how the state of a system will change over time for complex systems with millions of state variables and relationships. Advancing the algorithms for such inference so that ever larger models can be processed more quickly is likely to remain a crucial capability for engineering and science. Besides simulation, there is an increasing role for model checking, especially for engineered systems that are affected by high-impact low-probability events.

This leads to the following questions for discussion:

•   What are current trends in computing affecting modeling and simulation and how can they best be exploited?

•   How will these trends change the nature of simulation and reasoning algorithms?

•   What are the major gaps in computational methods for modeling and simulation, and what are the most important research problems?

•   How can one best exploit the vast amounts of data now becoming available to synergistically advance modeling and simulation for engineering complex systems?

2.3   Fidelity Issues and Uncertainty in Modeling and Simulation

The goal of modeling and simulation often is to make predictions, either to support decisions in an engineering, business or policy-making context, or to gain understanding and test hypotheses in a scientific context. It is impossible to prove a model is correct — the predictions are always uncertain. Yet, many models and simulations have been proven to be useful, and their results are routinely used for many purposes. To further improve the usefulness of models, it is important that we develop a rigorous theoretical foundation for characterizing the accuracy of the predictions. Within the modeling and simulation community, there is still a lack of agreement on how best to characterize this uncertainty. A variety of frameworks have been proposed around concepts of validation and verification, and a variety of uncertainty representations have been proposed, many based on questionable mathematical foundations (e.g., fuzzy numbers). A related question concerns determining the appropriate level of detail that a model should contain, and issues combining models that may operate at vastly different temporal or spatial scales.

This leads to the following questions for discussion:

•   What is the most appropriate approach to representing and reasoning about uncertainty in complex systems consistently?

•   What is the best approach to characterizing the uncertainty associated with a simulation model in order to enable and facilitate reuse?

•   How should one aggregate knowledge, expertise, and beliefs of multiple experts across different domains?

•   What is the best approach to take advantage of the large and diverse datasets for characterizing uncertainty and for improving model accuracy?

•   What are the most promising approaches to accelerate the validation of models for specific application contexts?

•   What are the key challenges in multi-resolution modeling and the most promising approaches to addressing them in the targeted application domains.

2.4   Model Reuse, Composition, and Adaptation

Although modeling has become indispensable in engineering and science, the cost of creating a good model can be considerable. This raises the question of how these costs can be reduced. One approach is to encode domain knowledge into modular, reusable libraries of models that can then be specialized and composed into large models. Such a modular approach allows the cost of model development, testing, and verification to be amortized of many (re)uses. However, reuse also introduces new challenges:

•   How can a model user be confident that a planned re-use of the model is within the range of uses intended by the model creator?

•   How can one characterize the uncertainty of a model that is reused (possibly with some adaptations to a new context)?

•   How can one characterize the uncertainty of simulation models obtained through composition of multiple models?

•   How can one accelerate the process of adapting and reusing models for different purposes? What are the fundamental limitations of technologies for model reuse?

2.5   Selected Applications Benefiting from Advances in Modeling and Simulation

Engineered systems continue to grow in complexity and scale. Existing modeling and simulation capabilities have not kept pace with the need to design and manage new emerging systems. Although the workshop is focused on modeling and simulation per se, distinct from the domain in which the technology is applied, the requirements of modeling and simulation technologies are ultimately derived from the application. In this context the workshop will explore new emerging developments in specific applications of societal importance in order to assess the needs and impacts that advances in modeling and simulation will have within those domains.

Specific application domains targeted by the workshop include:

•   Aerospace

•   Healthcare and medicine

•   Manufacturing

•   Security and defense

•   Sustainability, urban growth and infrastructures

3.   Expected Outcomes

The output produced by the workshop shall be a report documenting the observations and findings produced by its participants. The report shall be made available through the National Science Foundation. In addition, broad dissemination of the report is anticipated, e.g., through publications of summaries of the report as well as the report itself in professional journals and other publication venues. The ACM Transactions on Modeling and Computer Simulation (TOMACS) journal has expressed initial interest in publishing a summary of the workshop report, and Springer-Verlag has similarly expressed interest in publishing the full workshop report.

Longer term, further dissemination, discussion, and elaboration of workshop results are expected through follow on meetings and other activities associated with conferences and other activities in order to engage the broader modeling and simulation community. The National Modeling and Simulation Coalition (NMSC) which initiated discussions leading to this workshop and sponsored various related events leading up to this workshop will assist in disseminating workshop results, and working with groups such as the Modeling and Simulation Congressional Caucus in the United States to build broad support for this initiative.

4.   Workshop Logistics and Administration

A two-day workshop is planned to take place on January 13-14, 2016 at the National Science Foundation in Arlington Virginia. Participation is by invitation only. We anticipate approximately 40-50 attendees.

The bulk of the workshop will be spent in break out groups and cross-group discussions aimed to define research directions and articulate their importance in each of the five areas defined earlier. The workshop may also include a small number of presentations to set the stage for discussions.

Two individuals will be designated as the “leads” for each of the five areas. The main responsibility of the leads will be to coordinate and organize the writing of a chapter of the workshop report.