Aurora: A Parallel and Distributed Simulation System for Desktop Grids

Quick Navigation   1. Overview   2. Papers   3. Background   4. Architecture   5. API   6. Documentation   7. Download   8. Installation   9. Configuration  10. License

Aurora Borealis as seen from the Space Shuttle Atlantis during STS-117 (Credit: NASA)

Overview

Distributed computing with respect to donating idle processing cycles and volunteer computing have become well-received in recent years due to the success of projects which began with the Great Internet Mersenne Prime Search and evolved with distributed.net and SETI@home. The popularity of these efforts has expanded to include Graphics Processing Units (GPUs) and even video game consoles such as Folding@Home for the Sony PlayStation®3. These projects utilize spare processor cycles or share processor time at the user's discretion to execute portions of a larger task.

Traditional Parallel Discrete Event Simulation (PDES) systems are typically run on dedicated hardware and machines such as clusters and supercomputers. Although these platforms offer the highest performance for PDES applications, availability of these resources are restricted and usually highly competitive.

Organizations such as departments within a university or businesses have many desktop machines for general day-to-day use by students, staff, and faculty. Many of these machines may potentially lay idle for extended periods of time. Volunteer computing and desktop grids are able to allocate these resources for potential computations. This execution style is an attractive option for organizations due to being able to use existing machines and infrastructure.

The Aurora system attempts to bridge the gap between general distributed computing infrastructures such as Condor with the requirements of PDES codes. The Aurora system leverages a Master / Worker paradigm for work distribution similar to Condor MW but offers PDES-specific services such as Logical Process (LP) Management, Time Management, and Synchronization for message passing, data-parallel programs in contrast to Embarassingly Parallel (EP) applications such as Monte Carlo simulations.

Papers

1. Alfred Park, Richard M. Fujimoto, Parallel Discrete Event Simulation on Desktop Grid Computing Infrastructures, International Journal of Simulation and Process Modelling, Special Issue on Parallel and Distributed Simulation, Accepted for publication.
2. Alfred Park and Richard Fujimoto, Optimistic Parallel Simulation over Public Resource-Computing Infrastructures and Desktop Grids, in Proceedings of the 12th International Symposium on Distributed Simulation and Real-Time Applications (DS-RT) : IEEE Computer Society, Vancouver, BC, 2008.
3. Matthias Jeschke, Alfred Park, Roland Ewald, Richard Fujimoto, Adelinde M. Uhrmacher, Parallel and Distributed Spatial Simulation of Chemical Reactions, in Proceedings of the 22nd Workshop on Principles of Advanced and Distributed Simulation : IEEE Computer Society, Rome, Italy, 2008.
4. A. Park and R. Fujimoto, "A Scalable Framework for Parallel Discrete Event Simulations on Desktop Grids," in Proceedings of the 8th International Conference on Grid Computing: IEEE Computer Society, Austin, Texas, 2007.
5. R. Fujimoto, A. Park, and J. Huang, "Towards Flexible, Reliable, High Throughput Parallel Discrete Event Simulations," Modelling and Simulation Tools for Research in Emerging Multi-service Telecommunications, COST 285 Symposium, Surrey, UK, 2007.
6. A. Park and R. M. Fujimoto, "Aurora: An Approach to High throughput Parallel Simulation," in Proceedings of the 20th Workshop on Principles of Advanced and Distributed Simulation: IEEE Computer Society, Singapore, 2006. [DOI | ACM Portal]

Background

Discrete event simulations model a physical system through changes in state at distinct points in time. These changes are discretzied and ordered as events which are assigned time stamps. These events may change the state of the system and can generate new events. Other events may be randomly generated or generated apriori as specified by the application.

Parallel Discrete Event Simulations are simply discrete event simulations run across multiple processors which may span many machines. There are various reasons to parallelize a discrete event simulation: decrease runtime, increase model size, or perhaps integrate different models together to simulate multiple physical systems.

However, PDES inherently introduce more complexity into the system. Physical system processes are modeled as logical processes (LPs) within a PDES execution. These logical processes can communicate through a variety of methods such as shared memory abstractions and message passing. For the purposes of Aurora, we restrict LP communication to time stamp ordered messages which model physical process interaction. In other words, messages are discrete events which are passed between LPs for communication such as scheduling an event on a different LP (e.g., a packet being sent from one LP to another).

Typical PDES applications cover a wide range of sciences from telecommunication simulations, transportation simulations, to particle physics and biological applications.

Master/Worker Paradigm

In computing, the Master / Worker paradigm (henceforth referred to as MW) refers to the concept of a master entity controlling a pool of workers that are issued computations at the discretion of the master. MW typically lends itself well to volunteer computing frameworks and task-parallel jobs. In this style of parallel computation, typically classified as Embarassingly Parallel type of programs, these codes usually have no inter-process communication and can return simplified results where the size of the input is not representative of the return value or output of the program. This concept can also be used as motivation for volunteer computing and desktop grid systems, as in the case of the Aurora system, however, additional requirements must be met in order to distribute PDES codes across these infrastructures.

Desktop Grids

Desktop Grids are a specialized form of Grid and Volunteer computing. Desktop grids are a collection of machines within an organization such as a department within a university or a business which are loosely coupled together to form a computational resource. For the Aurora system, we view a desktop grid as nodes within an organization which may vary widely in architecture and speed, may be volatile in nature, and may not always be connected to the network. For instance, a desktop grid could consist of a computing lab, some thin clients, wireless notebooks, and desktop machines in faculty and staff offices. There are no guarantees that any of these machines will be available or connected at any time. Futhermore, these machines after becoming part of the computing resource pool may leave the resource pool at any time even if a computation is in progress.

The objective of the Aurora system is to explore the marriage of PDES and the desktop grid style of execution. There are many areas which the Aurora system delegates to lower layers (e.g., assumptions on infrastructure). For instance, security is delegated to system security inherent within the organization such as login and firewalls. In a trusted desktop grid environment, we also assume that computations are performed by clients which are non-malicious by nature. Each client is trusted and will return only good results (e.g., no Byzantine failures).

Clearly, this type of infrastructure is not optimal for PDES codes. PDES codes will always run at the highest efficiency on tightly-coupled systems with direct peer-to-peer communications such as clusters and supercomputing facilities. However, Aurora can provide additional and alternative means for providing computational resources for organizations employing PDES applications on potentially volatile machines which are shared or resources that may lay idle for extended periods of time.

Aurora Architecture

The Aurora architecture logically consists of three parts: the back-end system, clients, and simulation packages. The back-end system contains services which represent the logical master in the MW paradigm. The following sections detail each component.

Broker Service

The broker service provides master proxy host location query service for other services as well as providing multicast routing for fault-tolerance updates from the master proxy service.

Proxy Service

The proxy service is the logical master in the MW paradigm. The proxy service controls the metadata for all simulations, clients, and other services which make up the back-end system. Any number of proxy services can be instantiated where a single proxy is selected by the broker service as the master. The other proxies serve as slave proxies replicating metadata of the master proxy in case of master failure. The proxy service is made up of various managers which control specific portions of metadata to allow fine-grained access as low as a specific work unit within a simulation package which prevents blocking other requests which are unrelated.

The proxy contains a global view of the state of the simulation from the total work units leased, the progression of the simulation, lease deadlines, and which storage services are allocated for each particular simulation. In the case of a simulation rollback due to a fail over to a backup proxy service, the new master proxy may contain out of date data. However, it can simply discard any work unit returns that may conflict with its view of the simulation and may issue rollback requests to storage services.

State and Message Storage Services

The state and message services provide storage for simulation state and generated messages respectively. Storage for simulations are distinct services allowing scalability of the back-end by permitting the instantiation of any number and mixture of storage services to accomodate large-scale PDES codes which may store large state, generate many messages, or both.

The state storage service contains a state manager which stores a history of state updates. A history is required for both Conservative and Optimistic synchronization schemes to allow fault tolerance and rollbacks. In a conservatively synchronized simulation, there is no chance of rollback due to progress from the simulation, but rollback is possible through a service failure within the back-end system.

The message storage service contains a messages manager which stores time-stamp ordered priority queues of destination work units. These priority queues are effectively input queues for each work unit. The message manager is responsible for binning (e.g., message delivery) incoming messages from work unit updates to the proper destination work unit. Message leases are non-destructive and old messages are stored on the server until it is safe to reclaim memory with regard to fault tolerance.

For optimistic simulations, futher information must be maintained. For instance, even if the simulation has progressed past a work unit return, processed uncommitted messages (e.g., messages which have been processed by the client but is after Global Virtual Time) must be tracked in case of a simulation rollback. Handling of anti-messages or causality lists must be implemented as well for event cancellation.

Simulation Packages

Simulation packages is an interface to allow the definition of a PDES to be uploaded to the proxy service and to allocate resources such as state and message storage space. Example parameters that can be specified are total number of work units, simulation begin and end times, work unit wallclock deadlines, lookahead connectivity matrices, time management selection, and other requirement such as minimum client performance.

Once a simulation package is uplaoded to the proxy, back-end resources are immediately reserved so clients can begin computation immediately.

Client

Aurora clients provide the worker capability in the MW paradigm. Clients contact the master proxy service and register themselves receiving a unique ID to identify themselves for later transmissions and requests. Clients may register themselves as a pull-based (polling) client or a push-based (listening) client. Aurora supports a mixed-mode client operating environment where both pull- and push-based clients can operate simultaneously even on the same simulation package.

The Aurora client overlaps some communication tasks with computation as the simulation application is running. For instance if a message is generated during the simulation run, the destination work unit message service is located concurrently as the simulation application is running.

Clients perform a lifecycle of events continually until it is given a signal to stop. The first portion of the lifecycle leases the work unit to the client where the client contacts the proper back-end services to retrieve metadata, state, and messages. The state is unpacked by the application and messages are automatically queued in an internal priority queue to be fetched by the work unit. The second step initiates the simulation application-specific methods. The third and final step packages the current state variables and output messages and delivers them to the proper back-end services.

If the client fails during execution or leasing of metadata and simulation data, the lease is invalidated by the proxy and re-leased to a different client.

Runtime Libraries/Plugins

Aurora clients are generic execution platforms. In order to run the actual PDES code, the clients download specific libraries/plugins in the form of shared object (.so), dynamic libraries (.dylib), or dynamic link libraries (.dll). These plugins are dynamically loaded before simulation computation where the Aurora client instantiates a runtime class defined by the PDES application programmer.

This system allows clients to be deployed only once (per Aurora release) but enables them to execute any arbitrary simulation that is permitted to be uploaded to the back-end services. For example, a client could perform computation on SimPkg A for one work unit lease, but may execute a work unit from SimPkg B for the next lease. This allows for a unified client worker pool to operate over all simulation packages on the Aurora back-end instance.

Aurora Application Programming Interface (API)

Application programmers for the Aurora system will need to create two codes. The Simulation Package and the Runtime Library Plugin. The following sections will detail the interface available to application developers.

This API tutorial will highlight each interface and will outline a sample simulation package MySimPkg and its corresponding runtime library MyPlugin. For each method, Required or Optional is specified where if code must be present to properly overwrite and add functionality for the Aurora system to call. In either case, the method MUST be defined regardless if it is required or optional.

Simulation Package Interface

The Simulation Package Interface is used to create very simple programs to upload simulation parameters to the back-end services. Compiled programs are used instead of a web interface due to the necessity of creating lookahead connectivity matrices.

Header and Class Definition

The Simulation Package Interface is very simple to implement as a program. The first step is to include the AuroraSimPkg.h header and extend the Aurora::SimPkg interface. Two methods must be overwritten: CreateConnectionMatrix() and AppInitialize(). For Task-Parallel simulation packages, the CreateConnectionMatrix() method can simply return NULL. The AppInitialize() method is optional and may be left blank if no initialization needs to occur. This method, however, may be particularly useful for Task-parallel simulations. For example, it can be used to read in an input file to determine how to split up jobs into work units.

Runtime Plugin Interface

The Runtime Plugin Interface is where the simulation application programmer hooks simulation application code with the Aurora system. The simulation code can be self-contained within the entire runtime interface or it can simply call your existing code and act as a bridge.

Plugin Manifests

The Manifest block at the bottom of the class definition is very important. This is required so that Aurora/Poco is able to locate and instantiate the class to call methods defined apriori such as AppRun(). The POCO_BEGIN_MANIFEST must correspond with the simulation type you are creating, either Aurora::Runtime::PDES or Aurora::Runtime::TaskParallel. The POCO_EXPORT_CLASS must match the class to be instantiated by Aurora; in this example it is MyPlugin which corresponds to the class extending the runtime interface. You have the option of defining the configuration setting, LibraryClassName, in the .ini/.properties file of the simulation package. If this is not defined in the configuration file, then the FIRST exported class will be used, therefore, ensure that only ONE class is exported to the client in the manifest section. The PDES and TaskParallel examples below show how to integrate manifests into the class definition.

A full listing of available public API calls to the AuroraClient is available on this documentation page.

PDES Plugins: Header and Class Definition

PDES Simulation Loop Control

Aurora clients will have lease execution windows with simulation begin and end times each time a work unit is leased to them by the proxy service. The simulation application should "loop" on the execution window until it can no longer safely process events. This loop control is provided by the client interface. The above MyPlugin example illustrates how this is accomplished.

The simulation loop is a while loop with the conditional statement evaluating mClient->CanRun(). This evaluates if the simulation can run another iteration or if it must halt execution due to terminating conditions such as reaching the end of the simulation execution window.

There can be some variation in what follows next, but generally a simulation will either check its local event list (if it has one) or checks the Aurora message queue for new incoming messages. In this example, we check the Aurora message queue for the next message timestamp with the method call mClient->GetNextMessageTimestamp(ts). Since there could be no messages in the message queue or messages could exceed the execution window, the return value of this method must be checked. If the return value is true, then we have a valid next message timestamp which, in this example, is stored in the ts variable.

We can then set our simulation time to the message time via mClient->SetSimTime(ts). After the simulation time has been set to the appropriate value, we can then perform whatever operations are needed on the message received which will be specific to the simulation application. Finally, we should increment the Aurora event counter for every event processed (be it from an Aurora message or an internal event) via mClient->IncEventCounter().

mClient->EndRun() is provided to allow the application to terminate the loop after exhausting all messages in the incoming message queue and processable local events.

PDES Simulation Messaging: Retrieving Messages

There are three ways to retrieve the next incoming message for the work unit depending upon how much information about the message you require. mClient->GetNextMessage() is an overloaded method and the prototypes can be found on the client documentation page. Messages should be deserialized if they have been serialized before sending (strongly recommended for correct behavior across heterogeneous machines). See the serialization / deserialization reference below within this API section for more information.

PDES Simulation Messaging: Sending Messages

To send a message to another work unit (or even the same work unit), one can call the mClient->SendMessage() method. Messages should be serialized before sending so that they may be received across heterogeneous architectures. See the serialization / deserialization reference below within this API section for more information. It is important to note that the data buffer passed to SendMessage() is automatically enqueued for garbage collection by the Aurora system. It is not necessary to free this memory on your own.

Task-Parallel Plugins: Header and Class Definition

Task-Parallel runtime class definitions are similar to PDES, except that since an external program is used for the actual simulation, there is no AppRun() method. Moreover, instead of Initialize and Uninitialize, task-parallel jobs execute AppPreRun() and AppPostRun() code for any preparation code that needs to be executed and any cleanup code, respectively.

Serializer and Deserializer Helper Classes

The Aurora system includes easy-to-use serializer and deserializer classes to help the application programmer quickly package data for export across potentially heterogeneous machines. These classes utilize stream operators << (for serialization) and >> (for deserialization). For a full example on how to utilize these helper classes, see the Test sample in the samples/test directory.

For convenience, you can declare that you will be using the Aurora::Common namespace at the top of your C++ file. This will allow direct access to the Serializer and Deserializer classes:

Serializer

The serializer class is used to package data for export across the network to the back-end services and eventually to other clients which may be reside on a different operating system and architecture. Suppose we have 4 variables we would like to serialize:

To serialize these variables, we would do the following:

To export the serialized data as a buffer (e.g., for use with AppPackState() or SendMessage(), we can call the Export() method:

The data void pointer is a pointer to malloc'd memory holding the serialized variables while the size variable is written with how many bytes big the data buffer is.

Deserializer

The deserializer class can take in any data buffer packaged by the serializer class to extract data back out using the convenient streams interface.

The previously serialized values will be restored to the variables x, y, z, and str. It is important to note that with most serialization and marshalling schemes, the serialization and deserialization order must match. In other words, you must serialize and deserialize in the exact same order!

Makefiles and the Aurora Build System

Note that this section only applies to Non-Windows systems. For Windows, you can use existing solution / project files in the samples directory as a template for new projects.

Makefiles are very simple to create for Aurora plugins and simulation packages. Most of the logic resides in an application independent, universal makefile. The following Makefile is from the Test sample in the samples/test directory:

• BASEDIR is the base Aurora directory relative to the current path. Since the test sample resides in samples/test, the base Aurora directory is two levels up.
• PLUGIN is the name of the Plugin to compile. The final runtime library filename will be lib$(PLUGIN).so.$(PLUGIN_VERSION), $(PLUGIN).dylib.$(PLUGIN_VERSION), or $(PLUGIN).dll depending upon the platform.
• PULGIN_C_SRC declares any C plugin source files.
• PULGIN_CPP_SRC declares any C++ plugin source files.
• PULGIN_VERSION is the version number for the plugin which may be appended to the filename.
• SIMPKG is the simulation package name.
• SIMPKG_SRC declares any C++ source files for the simulation package.

Make sure to include the Aurora2.mk file and have the environment variable POCO_BASE set to your Poco installation directory.

Documentation

You can view the HTML browseable documentation of the Aurora2 system online. Alternatively, an offline documentation package exists in the download section. The documentation can be built from the sources if you have doxygen and graphviz installed via make doxygen-doc.

Download

By downloading the Aurora sources below, you agree to the terms set forth by the GTRC General Public License. The Poco library is licensed under the Poco/Boost Software License.

General Access Files

All of the Poco library distributions (Economy, NetSSL, Enterprise) are compatible with the Aurora2 software. The economy package should work fine for most, unless you wish to extend the Aurora2 software to add SSL or database functionalty.

Restricted Access Source Files

Changes and Release Notes

Installation Instructions

For the following Build instructions, it is assumed that $HOME points to your home directory. You may use whatever directory that fits you best.

Building Poco on Non-Windows Systems

Decompress Poco:

You will need to set an environment variable called POCO_BASE. The syntax for setting environment variables depends on your shell. For bash shells:

For csh:

You should add the above export/setenv POCO_BASE environment variable to your shell rc file (e.g., .bashrc, .cshrc, etc) so that you do not have to repeatedly set this value upon login when you need to compile Aurora programs.

Compile the libraries:

Building Poco on Windows Systems

On Windows, this example will decompress Poco to C:\lib. The sample code Microsoft Visual C++ solution and project files assume this path, however, you can change this to whatever fits your needs.

Decompress Poco:

Now you need to add the Poco bin directory to your envirnoment PATH variable. Go to Start/Settings/Control Panel/System. Select the Advanced Tab, then click the Environment Variables button. Add the following to the PATH variable:

Again, if you have installed Poco in a different directory use that path instead of C:\lib.

Open a Microsoft Visual Studio Command Prompt. Compile the libraries:

If you are not using Microsoft Visual Studio 2003 or higher, use the build_vs71 command instead.

Building Aurora2 on Non-Windows Systems

For this build process, this example will untar the Aurora source tarball to $HOME. Obviously you can decompress the source to any directory you choose.

Resulting binaries for the backend system and the client will reside in the bin directory.

Building Aurora2 on Windows Systems

If you have installed Poco to a directory other than C:\lib\poco-1.3.1, you will need to update all of the .vcproj files to reflect the proper Poco directory. Windows Vista users can create symbolic links (junctions) linking their chosen install directory to C:\lib\poco-1.3.1 so that the default .vcproj files can be used.

With Microsoft Visual Studio 2003 or higher, open a Microsoft Visual Studio command prompt.

This batch file will automatically create the Aurora services from the solution and project files. If you do not have a Microsoft Visual Studio version that supports solution files, individual backend components may be compiled under the msvc directories within each main service directory under the src folder using the .vcproj files.

Resulting binaries for the backend system and the client will reside in debug and release directories. Use whichever binaries you require (e.g., debugging or deployment/performance respectively).

Building the GUI Simulation Package Manager

Note that this is a completely optional part of the Aurora system. It does not need to be built in order for the system to function. Pre-built GUI binaries are available in the download section, but there are no guarantees that they will work for your system.

In order to build the GUI from source, you will need to have at least wxWidgets 2.8 with development libraries installed. If you are compiling under Linux, you will also need GTK+ installed as well. Installation of these libraries is outside the scope of this document. However, the wxWidgets Wiki has installation guides available for each platform. Note that under Windows, it is assumed that the wxWidgets toolkit is installed in C:\lib.

To build the GUI under non-Windows systems, issue the make command:

Under Windows, load up the solution or project file in the msvc folder in util/gui, ensure that the include paths and library dependencies are correct, and build.

Sample Simulations

Samples are found in the samples directory. For non-Windows sytems, simply type make in the sample directory. For Windows, there is a msvc directory within each sample directory which contains solution and vcproj files.

There are two sample simulations included in the general source distribution of Aurora2:

1. Test: Small Test parallel discrete event simulation. For validation of the Aurora2 install. Also shows how to use the Serializer and Deserializer helper classes.
2. Queue: Torus Queuing Network simulation. Uses input files and aggregates simulation logical procesess into larger Aurora work units. Note that this program was a port from the previous Aurora releases and does not use the Serializer and Deserializer classes as recommended for portability.

This example execution of the test simulation sample assumes a *nix style environment. Procedure under Windows would be similar.

To run a simulation, we first need to instantiate the back-end services. For convenience, it will be easier to have 5 shells (or 5 screen sessions) open. Alternatively, you can opt to have a single shell by utilizing the built-in daemonizing switch (--daemon) to background the back-end services, but you may not be able to see the output of these services.

Sample configuration .ini files exist in the util/ini directory. Copy these files to the bin directory and modify them to fit your system as specified in the configuration section. After proper configuration, start the services (residing in the bin directory) in order: a2broker, a2proxy, a2statesrv, a2msgsrv.

The a2client.ini and TestSimPkg.ini files in the samples/test directory should also be modified before execution. You need to ensure that your OS and architecture platform are specified in the [Library] section of the TestSimPkg.ini file. To query your system for the OS-Architecture string, simply run any service with the -h (or /h in Windows). You will see a "Detected System" line. For example:

Ensure that this identifier string exists in the TestSimPkg.ini file mapped to the proper shared object (.so), dynamic library (.dylib), or dynamic link library (.dll) file. For the example above, we are running a 64-bit version of Linux thus our library mapping is Linux-x86_64 as specified by Aurora. Note that you can specify multiple mappings under [Library] to add additional client architectures that can be supported by this simulation package (see the client configuration section for more information). For now, however, we are just interested in running this sample under our single client on the one machine, thus the configuration setting would be:

To build the test sample plugin and simulation package, follow these steps:

Once these ini files have been properly configured and you have built your application plugin and SimPkg uploader (via make), you can upload your simulation package definiton to the back-end services via:

You should receive feedback indicating that the simulation package definition was uploaded successfully:

Now that the back-end has been prepared with the necessary simulation parameters, you can start your client to run the simulation. Ensure that the Broker host address and port are correct in the a2client.ini file. Assuming we are still in the samples/test directory, we can simply run the client as:

This client invocation from the samples/test directory works because the a2client.ini files resides locally. If the a2client.ini file was somewhere different, we could issue the -c (or /c on Windows) command-line switch to specify where the a2client.ini file resides.

You should now see the client executing work as directed by the proxy service. If the client hangs or crashes attempting to load the application plugin, you have incorrectly specified your LibraryClassName, exported multiple classes in the plugin manifest, or there is a Library platform mismatch in the simulation package defintion.

Once the iteration count reaches 7/0 for the test simulation, the simulation has completed. Simply break out of the client via CTRL-C, and you should see final performance statistics. To validate your install, make sure you get identical total events processed, total state exported, and total messages exported as the following:

Configuration System

Aurora2 uses .ini/.properties files to configure services in addition to reading in parameters from the command-line. Sample ini files for the backend services can be found in the util/ini directory. These configuration files should reside in the same directory as the service, or the proper path to the configuration file can be given via the command-line switch -c (non-Windows) or /c (Windows).

Please take special note of which configuration blocks are required.

Creating Logfiles

All services with the exception of simulation packages offer logging to files. To utilize the file logging system, one can specify the logfile either on the command-line via the -l (or /l in Windows) switch. Alternatively, a logfile can be specified in the [Configuration] block of a service ini file:

Sometimes it may be more convenient not to modify ini/properties files to capture output to a file (e.g., spawning many clients via a shell script). To redirect output to a file, ensure that the Standard Error stream is redirected. For example, to capture both standard out and standard error to a file:

Specifying the Broker Address

This configuration block is required for the a2proxy, a2statesrv, a2msgsrv, a2client, and simulation packages.

• IP specifies the IP address or FQDN hostname of the broker service. Avoid using loopback addresses unless everything is contained on one node.
• Port specifies the port of the broker service

Broker Service Configuration

This configuration block is optional, but recommended.

• Port specifies the port to bind the broker service to.
• Threads specifies the number of service threads in the general worker pool.
• ProxyHeartbeat is the number of milliseconds to wait between heartbeats to proxies for fault tolerance purposes.
• IPAliases [optional] is a comma-delimited list of IP aliases. In this example, the IP 1.2.3.4 will be replaced with 5.6.7.8. 10.11.12.13 will be replaced with 20.21.22.23. This is particularly useful for machines which have multiple interfaces and you need to force communication through a public interface rather than a private one.

Proxy Service Configuration

This configuration block is optional, but recommended for the a2proxy.

• DistributeWorkWaittime is the time interval in milliseconds between pushing leases to push-based clients.
• LeaseExpireCheckWaittime is the time interval in milliseconds between sweeping client lists for expired leases to release back to the available work pool.
• FaultToleranceWaittime is the time interval in milliseconds between fault tolerance updates. A value of zero disables the fault tolerance system.
• FaultToleranceSaveToDisk specifies whether to save fault tolerance data to disk.
• IPAliases [optional] is a comma-delimited list of IP aliases. In this example, the IP 1.2.3.4 will be replaced with 5.6.7.8. 10.11.12.13 will be replaced with 20.21.22.23. This is particularly useful for machines which have multiple interfaces and you need to force communication through a public interface rather than a private one.

State and Message Storage Service Configuration

This configuration block is optional, however, it is recommended to specify the entire configuration block. This block is required if you wish to utilize the fault-tolerance system.

• Memory is the maximum amount of memory in MiB the server will have available for work units.
• Backup specifies a preference on whether this server should be designated as a backup server for fault tolerance purposes. The actual designation will be determined by the proxy service (iff there do not exist enough primary servers will a backup be registered as a primary server).
• FaultToleranceWaittime is the time interval in milliseconds between fault tolerance updates. A value of zero disables the fault tolerance system.
• FaultToleranceSaveToDisk specifies whether to save fault tolerance data to disk.

Client Service Configuration

Some entries in the Client general configuration block are optional, however, it is recommended to specify the entire configuration block.

• Port [required] specifies the listening port for the client service. If this value is zero, no listening port is created and instead the client becomes a pull-based (polling) worker.
• SimPkgID [optional] specifies the simulation package to "bind" to as a worker. If not specified, this client becomes a generic worker for the entire Aurora backend instance which will perform work on any available workunit regardless of simulation package. Note that if the SimPkg ID is specified, the client will auto-terminate once the simulation reaches completion.
• Memory [optional] specifies the amount of memory in MiB that is available to the client. A value of zero disables memory requirement checking.
• Disk [optional] specifies the amount of disk space in MiB that is available to the client. A value of zero disables disk space requirement checking.
• Iterations [optional] specifies the maximum number of times a client can download work units before exiting. A value of zero allows the client to loop forever.
• WhetstoneTime [optional] specifies the amount of time in seconds to spend performing a Whetstone benchmark of the client. The default value is 30 seconds.
• DhrystoneTime [optional] specifies the amount of time in seconds to spend performing a Dhrystone benchmark of the client. The default value is 30 seconds.

The following additional client configuration blocks are used for Task-Parallel clients only for specifying capabilities. The following blocks are required for Task-Parallel simulations.

• Capability is the name of the required resource that this client can provide.
• Program is the exact location of the capability. If spaces are included in the path, ensure that the string is enclosed in quotation marks.
• Arguments are the default arguments to pass when opening the program.
• Versions [optional,comma delimited list] specifies the versions of the capability installed on this client. Each version requires a path mapping in the [Versions] configuration block.

• V2.0 / V3.2 are the mappings of the of the capability version to its path on the machine. Of course V2.0/V3.2 are used in this example but any string can exist here that maps to the previous "Versions" string in the [Capability] configuration block.

Simulation Package Configuration

The following configuration block is required for both PDES and Task-Parallel simulation package uploaders.

• Type specifies the simulation package type. This value can be either PDES or TASK-PARALLEL.
• ID specifies the unique simulation package identifier. This value must be unique to the entire Aurora backend instance.
• Memory is the amount of memory in MiB required per work unit.
• Deadline specifies the work unit return deadline in seconds.
• WorkUnits specifies the number of work units in the simulation (or logical processes if there is a 1:1 mapping between LPs and WUs).
• Dhrystone specifies the required client Dhrystone score.
• Whetstone specifies the required client Whetstone score.
• LibraryClassName [optional] specifies the class to instantiate in the runtime library/plugin to execute application-specific code. If this is not specified, the client will iterate through the manifest and assign the first available class to be instantiated. Therefore, ensure that only ONE class is exported for the client to instantiate.
• InputFiles [optional] is a comma-delimited list of input files to upload to the proxy. These files can be used by the application to load more configuration parameters or for any other conceivable use for input files to a simulation.

The following additional configuration options for the [SimPkg] block applies to PDES simulation packages only.

• TimeManagement specifies which time management scheme to employ.
• BeginTime specifies the simulation begin time.
• EndTime specifies the simulation end time.

The following additional configuration options for the [SimPkg] block applies to Task-Parallel simulation packages only.

• Capability specifies what capability name this simulation package requires.
• Arguments [optional] are additional simulation package-specific arguments that can be passed to the program.
• OutputFiles is a comma-delimited list of output files to capture by the client for collection.
• RequiredVersion is the version of the capability required.

The following block is required for both PDES and Task-Parallel simulation packages to specify runtime libraries/plugins for application compatible platforms. These OS-Architecture identification strings are pulled from the Poco System Configuration utility. You can determine what OS-Architecture string your machine is on by simply running a service with the -h (non-Windows) or /h (Windows) switch. The service will print out the detected system string, however, do not provide any versioning information enclosed in brackets []. For more information, see the samples section above.

The above example block configures the simulation package for Linux 32-bit and Windows NT/2K/XP/Vista 32-bit platforms. Of course when uploading your simulation package, ensure that these 2 files are available with the simulation package uploader so that they may be uploaded along with the simulation parameters to the proxy service.

Note that is possible to cross-compile your plugins for most platforms that GCC supports. This can be simplified via CrossTool. For an in-depth look at how to cross-compile on Poco, see this wiki page.

Georgia Tech Research Corporation (GTRC) General Public License

GENERAL PUBLIC LICENSE AGREEMENT

BY UTILIZING THIS PROGRAM, YOU AGREE TO BECOME BOUND BY THE TERMS OF THIS LICENSE. IF YOU DO NOT AGREE TO THE TERMS OF THIS LICENSE, DO NOT USE THIS PROGRAM OR ANY PORTION THEREOF IN ANY FORM OR MANNER.

This Program is licensed, not sold to you by GEORGIA TECH RESEARCH CORPORATION ("GTRC"), owner of all code and accompanying documentation (hereinafter "Program"), for use only under the terms of this License, and GTRC reserves any rights not expressly granted to you.

1. This License allows you to:

(a) make copies and distribute copies of the Program's source code provide that any such copy clearly displays any and all appropriate copyright notices and disclaimer of warranty as set forth in Article 5 and 6 of this License. All notices that refer to this License and to the absence of any warranty must be kept intact at all times. A copy of this License must accompany any and all copies of the Program distributed to third parties.

A fee may be charged to cover the cost associated with the physical act of transferring a copy to a third party. At no time shall the program be sold for commercial gain either alone or incorporated with other program(s) without entering into a separate agreement with GTRC.

(b) modify the original copy or copies of the Program or any portion thereof ("Modification(s)"). Modifications may be copied and distributed under the terms and conditions as set forth above, provided the following conditions are met:

i) any and all modified files must be affixed with prominent notices that you have changed the files and the date that the changes occurred.

ii) any work that you distribute, publish, or make available, that in whole or in part contains portions of the Program or derivative work thereof, must be licensed at no charge to all third parties under the terms of this License.

iii) if the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to display and/or print an announcement with all appropriate copyright notices and disclaimer of warranty as set forth in Article 5 and 6 of this License to be clearly displayed. In addition, you must provide reasonable access to this License to the user.

Any portion of a Modification that can be reasonably considered independent of the Program and separate work in and of itself is not subject to the terms and conditions set forth in this License as long as it is not distributed with the Program or any portion thereof.

2. This License further allows you to copy and distribute the Program or a work based on it, as set forth in Article 1 Section b in object code or executable form under the terms of Article 1 above provided that you also either:

i) accompany it with complete corresponding machine-readable source code, which must be distributed under the terms of Article 1, on a medium customarily used for software interchange; or,

ii) accompany it with a written offer, valid for no less than three (3) years from the time of distribution, to give any third party, for no consideration greater than the cost of physical transfer, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Article 1 on a medium customarily used for software interchange; or,

3. Export Law Assurance.

You agree that the Software will not be shipped, transferred or exported, directly into any country prohibited by the United States Export Administration Act and the regulations thereunder nor will be used for any purpose prohibited by the Act.

4. Termination.

If at anytime you are unable to comply with any portion of this License you must immediately cease use of the Program and all distribution activities involving the Program or any portion thereof.

5. Disclaimer of Warranties and Limitation on Liability.

YOU ACCEPT THE PROGRAM ON AN "AS IS" BASIS. GTRC MAKES NO WARRANTY THAT ALL ERRORS CAN BE OR HAVE BEEN ELIMINATED FROM PROGRAM. GTRC SHALL NOT BE RESPONSIBLE FOR LOSSES OF ANY KIND RESULTING FROM THE USE OF PROGRAM AND ITS ACCOMPANYING DOCUMENT(S), AND CAN IN NO WAY PROVIDE COMPENSATION FOR ANY LOSSES SUSTAINED, INCLUDING BUT NOT LIMITED TO ANY OBLIGATION, LIABILITY, RIGHT, CLAIM OR REMEDY FOR TORT, OR FOR ANY ACTUAL OR ALLEGED INFRINGEMENT OF PATENTS, COPYRIGHTS, TRADE SECRETS, OR SIMILAR RIGHTS OF THIRD PARTIES, NOR ANY BUSINESS EXPENSE, MACHINE DOWNTIME OR DAMAGES CAUSED TO YOU BY ANY DEFICIENCY, DEFECT OR ERROR IN PROGRAM OR MALFUNCTION THEREOF, NOR ANY INCIDENTAL OR CONSEQUENTIAL DAMAGES, HOWEVER CAUSED. GTRC DISCLAIMS ALL WARRANTIES, BOTH EXPRESS AND IMPLIED RESPECTING THE USE AND OPERATION OF PROGRAM AND ITS ACCOMPANYING DOCUMENTATION, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR PARTICULAR PURPOSE AND ANY IMPLIED WARRANTY ARISING FROM COURSE OF PERFORMANCE, COURSE OF DEALING OR USAGE OF TRADE. GTRC MAKES NO WARRANTY THAT PROGRAM IS ADEQUATELY OR COMPLETELY DESCRIBED IN, OR BEHAVES IN ACCORDANCE WITH ANY ACCOMPANYING DOCUMENTATION. THE USER OF PROGRAM IS EXPECTED TO MAKE THE FINAL EVALUATION OF PROGRAM'S USEFULNESS IN USER'S OWN ENVIRONMENT.

GTRC represents that, to the best of its knowledge, the software furnished hereunder does not infringe any copyright or patent.

GTRC shall have no obligation for support or maintenance of Program.

6. Copyright Notice.

THE SOFTWARE AND ACCOMPANYING DOCUMENTATION ARE COPYRIGHTED WITH ALL RIGHTS RESERVED BY GTRC. UNDER UNITED STATES COPYRIGHT LAWS, THE SOFTWARE AND ITS ACCOMPANYING DOCUMENTATION MAY NOT BE COPIED EXCEPT AS GRANTED HEREIN.

You acknowledge that GTRC is the sole owner of Program, including all copyrights subsisting therein. Any and all copies or partial copies of Program made by you shall bear the copyright notice set forth below and affixed to the original version or such other notice as GTRC shall designate. Such notice shall also be affixed to all improvements or enhancements of Program made by you or portions thereof in such a manner and location as to give reasonable notice of GTRC's copyright as set forth in Article 1.

Contact Information