Skip to content

Testing

Jump to bottom
Stephen Frank edited this page Oct 17, 2013 · 4 revisions

Testing the Library

The Test directory within the Campus Energy Modeling project contains a suite of scripts and models for the MATLAB/Simulink library which may be used to test development snapshots and release candidates.

Organization

The Test directory contains the following files and folders:

  • data Directory containing data files required by test scripts and demos
  • matlab Directory containing test scripts of Campus Energy Modeling MATLAB functions
  • simulink Directory containing test models and scripts for Campus Energy Modeling Simulink library blocks
  • run_tests.m Run script for execution of the full suite of tests (see Running the Tests)

Subfolders within the matlab and simulink directories contain the individual tests.

The file Info.md within the data directory documents the various data files and their original sources.

Running the Tests

Each test is self-contained and may be run individually by executing the MATLAB script located in the test folder. (If there is more than one script, execute the script with a name that matches the test folder name.) Be aware that the test will fail if the local machine is missing any required dependencies.

To automatically execute the entire suite of tests, edit and run the script run_tests.m located in the Test directory. The script will automatically discover and execute all available tests. Failed tests do not terminate execution but do generate log entries; see Logging.

Test execution may be customized by modifying the variable values assigned in the Configuration section at the top of the script, including limiting tests by category (MATLAB or Simulink) or by software dependency. For details, see the documentation within run_tests.m.

Dependencies

Most of the tests for the Campus Energy Modeling project require one or more software packages in addition to MATLAB and Simulink. Each individual test identifies its dependencies, if any, in the file deps.cfg. (If deps.cfg does not exist, then the test has no dependencies beyond vanilla MATLAB and Simulink.)

By default, run_tests.m executes all tests, including those with additional dependencies. To exclude tests with specific dependencies, add those dependencies to the cell array excludeDeps. For example,

excludeDeps = {'ssc','databus'};

excludes all tests which have either SSC or DataBus listed as a dependency. The documentation within run_tests.m includes a list of standard names for the various dependencies.

Logging

When run_tests.m executes, it writes a master log file which summarizes the result of each test. By default, this file is named run_tests.log and located in the Test directory; the name and location may be modified by assigning a different relative path to the logfile variable in the test configuration. If the log file already exists, the script appends to it rather than overwriting it.

In addition to the master log, run_tests.m also creates (or appends) log files in each individual test folder. The log file names match the parent test name, for example, ssc_pvsamv1.log for the test ssc_pvsamv1. Each log documents the start and end time of the test and records any MATLAB command line output generated by the test script, including error messages. If the configuration variable clearOldLogs is set to true, then each individual log file will be erased prior to executing the corresponding test.

Troubleshooting Failed Tests

The individual log file for a failed test contains an error trace that documents the line which generated the exception. Use this information as a starting point for troubleshooting. Some tests generate plots in addition to writing the log file. If a test fails, these plots may be examined for clues.

Developing New Tests

Philosophy

Unlike demos, which may demonstrate multiple features of the Campus Energy Modeling library, tests are designed to isolate and verify correct behavior for one library feature. In this context, a feature is

  • A single MATLAB function,
  • A collection of related MATLAB functions that provide a specific service, or
  • A single Simulink block.

Please create tests which address only one feature. (If your test relies on other features, ensure that separate tests exist for them.)

Second, the test script must be portable.

  • Use platform-independent code
  • Do not reference files with paths external to the Campus Energy Modeling project

Finally, the test must not require any user input or manual editing prior to execution.

Structure

To allow fully automated execution of the test suite via run_tests.m, each test must conform to several strict structural conventions.

  1. Place tests for the Campus Energy Modeling MATLAB functions in the matlab folder and tests for the Simulink blocks in the simulink folder. Tests involving a mixture of MATLAB functions and Simulink models are classified as Simulink tests.
  2. Each test must be self-contained: all required files must be included in the test folder (with the exception of data files from the data directory).
  3. Each test must execute via a single, pre-configured MATLAB test script. The name of this script script must match the name of the test folder. Example: ssc_pvsamv1.m is the script for test ssc_pvsamv1.
  4. Execute Simulink models programmatically from within the test script.
  5. To avoid ambiguity, do not name the Simulink model identically to the test script. Otherwise, run_tests.m may attempt to execute the Simulink model directly instead of running the test script.

Clone this wiki locally