Model Verification

Ensuring model accuracy and validity is a critical task and there are a number of procedures that may be carried out in order to help obtain confidence in your model results. These confidence-building activities include:

·         Historical Tests: Comparing the results of the model to known historical values.

·         Extreme Value Tests: Testing how the model behaves when extreme values are inputted into it. Does the model behave as one would expect or does it exhibit wild behavior?

·         Submodel Tests: For large models it is sometimes possible to isolate one portion of the model and compare it to some other more detailed model designed for analyzing that specific feature.

In addition to these methods, Insight Maker includes a rich suite of features designed for documenting the assumptions that your model is based on and ensuring that these assumptions are not violated during model simulation.

For instance, imagine you are designing a model simulating an ecosystem. As part of this complex model, you may have a stock representing the quantity of water in a lake. When designing your model, you know that the value of the lake stock should never become negative. If it does, there is clearly something wrong with your results and you should analyze the model to see what is causing this error. Unfortunately, though this constraint is fresh in your mind when you are focusing on the lake, it is very easy to forget about it when you focus on a different item. When you move on to another part of your model you might inadvertently make a change to your model that violates the assumption about the lake. Possibly you add another flow that results in the lake stock becoming negative and, since your focus distracted, you do not realize this mistake until much later.

In order to prevent mistake like these, Insight Maker includes a set of tools for documenting and enforcing your model’s assumptions as you develop them. These tools include: Unit Checking and Constraint Checking.  All of these tools can be accessed by clicking the model verification button that appears in the configuration pane when you select a stock, flow, converter, or parameter within your model. Once you have defined verification checks for a primitive, a checkmark will appear on the primitive’s verification button letting you know that primitive will be validated during the simulation.

Constraints

Insight Maker constraints allow you to very simply set the maximum and minimum allowable values for a primitive. Thus, in an example of a lake, we would set a minimum value of 0. For a variable representing a proportion, the value could range from 0 to 1.

If the constraints are ever violated for a primitive during the simulation, an error message will be displayed and the simulation will be terminated.


Units

Units are at the heart of Insight Maker. The flows into a stock should have the same units as the contents of that stock. Similarly, you should be able to specify the units for the different parameters and converters in your model. This is an underlying assumption of System Dynamics models, and it is one that you should adhere to as you construct your models. If you do construct models with disregard to units, it is still possible to develop accurate simulations, however it is much more likely that you will create errors or your model will fail to reflect the underlying processes driving reality.

Insight Maker allows you to enforce units using its unit mechanisms. As you construct your model, you may define the units for the different parts of your model. For instance, you may add a stock representing a lake with units of Cubic Meters. You may have a river flowing into the lake with units of Cubic Meters per Second. When you run a simulation, Insight Maker will check to make sure that the units you have defined are consistent. If they are not consistent (for instance you try to subtract a number with units of Meters from one with units of Liters) Insight Maker will display an error and terminate the simulation.

When you define a number in Insight Maker, it is automatically defined as Unitless. When the value of a valued primitive (stock, flow, parameter, or converter) whose units are defined is referenced elsewhere in your model, Insight Maker will evaluate its equation. If the result of this evaluation is a Unitless number, Insight Maker will then assign the units of that primitive to the value. If, however, the value is not Unitless and the units conflict with the primitive’s units, then an error will be displayed. You may add and multiply Unitless numbers by and to numbers with defined units and the results will take on the units of the number for which units were defined. You may also define a number with specified units in your equations using a special, curly-bracket notation like follows:

{2 Meters} + {10 Centimeters}

The above equation defines a value of 2.1 meters. Insight Maker has the ability to carry out automatic unit conversion between certain sets of units. Thus if you subtract a value of 10 centimeters from a value of 2 meters the result will be 190 centimeters or 1.9 meters.  The following table illustrates some examples of unit operations within Insight Maker:

Equation

Results

1+3

4 (No Units)

{1 Centimeter}+{3 Centimeters}

4 Centimeters

{1 Meter}+{3 Centimeters}

1.03 Meters

{1 Grams}+{3 Centimeters}

Error – Inconsistent Units

1+{3 Centimeters}

Error – Inconsistent Units

2*7

14 (No Units)

2*{7 Meters}

14 Meters

{2 Meters}*{7 Meters}

14 Square Meters

{2 Meters}/{4 Seconds}

0.5 Meters/Seconds

{2 Meters/Seconds}/4

0.5 Meters/Seconds

{25 Square Meters}^(1/2)

5 Meters

3*{200 Centimeters}*{4 Square Meters}

24 Cubic Meters

{100 Inches} – {1 Meters}

154 Centimeters

{10 Meters/Second}*{1 Minute}

600 Meters

{7 Widgets/Years^2}*{10 Years}

70 Widgets/Year

Insight Maker comes with a large number of units built into it. However, it also contains the ability for you to define custom units, like "Widgets" above, by simply typing their name into the unit field for a primitive.