27.3 More Testing for Developers - DOES NOT WORK 2001

This section is for heavier testing than above, say of a new version of NCGB.1

27.3.1 Setting the Testing Environment

The file TestingEnvironment.m has four commands that may require editing. This file basically defines the directories where the testing is being done, where the sample problems are stored, where the results from the test are outputed and where the trusted reference answers are kept.

The command $NC$TestPrefix$ defines where the testing is being done. Its default value is  /NC/NCGB/Testing/, and should not be monkeyed with unless you have created your own testing directory.

$NC$TestInput$ defines where the test problems are kept. If you have some test problems of your own, you may test them by giving the path to them as the value. The default is

$NC$TestInput$ = ”StringJoin[$NC$TestPrefix$,”TestProblems/”]

If you do several tests, you can keep the results in separate directories by giving

$NC$TestResults$ the path as its value.

The default setting is

$NC$TestResults$ = ”StringJoin[$NC$TestPrefix$,”TestResults/”]

$NC$TestReferences$ defines the directory in which the trusted answers to the test problems are stored. When you run the test, the files in TestResults will be compared to these reference files. Also if you are creating your own reference files, you may want to redefine the path to avoid overwriting the answer we so helpfully provide. The default is

$NC$TestReferences$ = ”StringJoin[$NC$TestPrefix$,”TestReferences/”]

Remember, before doing anything, make sure the TestingEnvironment.m file has the right definitions to avoid sending files in all directions.

Creating Test Results

To create a set of test outputs stay in the Testing directory. Make sure you have lots of test problem files in the TestProblems directory; say c01.data.m thru c85.data.m. Make sure the TestReference directory is loaded with files generated by applying a version of NCGB that you trust to the problems in TestProblems. This should be the case because the package NCGB is shipped with such a collection of sample answers.

Edit 2 lines in the file

init.TestResults.m

to put in the path to the version of NCGB that you want to test.

Copy the file

init.TestReferences.m
to the file init.m

Run     Mathematica.

Load    NCGBTestCreate.m

Run

NCGBTestCreate[Integer1, Integer2]

Exit Mma

Here the arguments Integer must be a non-negative integer, satisfying Integer1 Integer2 85 This creates a set of files of answers obtained from NCProcess labelled

cN.out.tex

where N is an integer between Integer1 and Integer2. They are stored in the directory TestResults.

If you want to test a program besides NCProcess you can use

NCGBTestCreate[Integer1,Integer2,FunctionName]

Label whichever program you want to test with FunctionName. For example, the name NCMakeGBFunction has already been assigned to NCMakeGB, so running

NCGBTestCreate[Integer1,Integer2,NCMakeGBFunction ]

will create files for testing NCMakeGB.

Comparing to Reference Code

While in the testing directory

Run

NCGBTestCompare[ Integer, Integer]    (* for *.tex files only *)

or to test programs other than NCProcess

NCGBTestCompare[ Integer, Integer, suffix]    

(* for example, suffix is ”GB” only — remember the quotes *)

Go get a cup of coffee. When you come back look at the resulting screen output.

You should get

test #01 True
test #02 True
etc.

The command NCGBTestCompare has created answer files and compared them to the old reference files.

Creating Reference Answers

Maybe you do not like the reference results we provided to you (some people are not very grateful). To create a new set of reference outputs stay in the Testing directory. Make sure you have lots of test problem files in the TestProblems directory; say c01.data.m thru c78.data.m.

Edit 2 lines in the file

init.TestReference.m

to put in the path to the version of NCGB that you want as a reference.

Copy the file

init.TestReferences.m to the file init.m

Run     Mathematica

Load    NCGBTestCreate.m

Run

NCGBTestCreate[Integer1, Integer2]

This is beginning to look familiar. From here on proceed as you did in creating test files. This creates a collection of files in the directory TestReference.