The goal is to test TWO versions of NCGB against each other to see if they produce exactly the same answer. Or one can test a version of NCGB against output which has been stored for reference. All files involved are in the subdirectory Testing of NCGB.
There are four directories inside Testing.
We do not trust our testing programs for NCGB much. They need work. The NCAlgebra test program is excellent.
There is an old testing program. We do not support or upgrade it but find it quite useful. It is GBTEST and the use of it is different as that described for NCGBTEST above. First you load the file GBTest.m . You run it by typing GBTEST[1,48] to run tests 1 thru 48, or for those with less patience GBTEST[12,28] to run tests 12 thru 28, etc.
Yet another class of specialized tests is run simply by loading the file NCGBXTEST99. These tests follow the format of the NCAlgebra tests NCTEST exactly. After you load the file
you hope to get True, True, printing to the screen. The tests in NCGBXTEST99 test programs which run under NCGB and were developed in 1999. They give NCGB a pretty good work out.
Someday we plan to integrate our testing procedures. As you see we now have a mixed bag of tests.
THE COMPAREING STEP DOES NOT WORK in 2001.
When you download NCGB from the net, the directories TestProblems and TestReference contain many files. Those in TestProblems supply mathematical exercises to the testing programs. Those in TestReference are the result of running NCGB in ideal circumstances.
To see if the version of NCGB you downloaded is complete and functioning load NCGBTEST, If you do not have write access to NC/NCGB/Testing you may have to change the variable $NC$TestResults$. TestingEnvironment.m defines the default depository for Test Result files. This is described in greater detail in Section 27.3.1.
Go get a cup of coffee or two cups of coffee or a good night’s sleep. When you come back look at the resulting screen output by typing
You should get
This tests your version of NCGB against answers obtained from the version we have. If they do not agree there is a problem (possibly minor). One must be very careful with NCGBTEST. If paths are set wrong, then it will find no files for either the reference answers or the new code you are testing and since these behaviors are identical NCGBTEST will return True, True, etc. which is of course False, False, False!
There is a command which acts globally to turn output on and off:
also the NCProcess option
turns on (and off) more diagnostics only inside NCProcess. Default on both of these is False.
This section is for heavier testing than above, say of a new version of NCGB.1
The file TestingEnvironment.m has four commands that may require editing. This file basically defines the directories where the testing is being done, where the sample problems are stored, where the results from the test are outputed and where the trusted reference answers are kept.
The command $NC$TestPrefix$ defines where the testing is being done. Its default value is /NC/NCGB/Testing/, and should not be monkeyed with unless you have created your own testing directory.
$NC$TestInput$ defines where the test problems are kept. If you have some test problems of your own, you may test them by giving the path to them as the value. The default is
If you do several tests, you can keep the results in separate directories by giving
The default setting is
$NC$TestReferences$ defines the directory in which the trusted answers to the test problems are stored. When you run the test, the files in TestResults will be compared to these reference files. Also if you are creating your own reference files, you may want to redefine the path to avoid overwriting the answer we so helpfully provide. The default is
Remember, before doing anything, make sure the TestingEnvironment.m file has the right definitions to avoid sending files in all directions.
Creating Test Results
To create a set of test outputs stay in the Testing directory. Make sure you have lots of test problem files in the TestProblems directory; say c01.data.m thru c85.data.m. Make sure the TestReference directory is loaded with files generated by applying a version of NCGB that you trust to the problems in TestProblems. This should be the case because the package NCGB is shipped with such a collection of sample answers.
Edit 2 lines in the file
to put in the path to the version of NCGB that you want to test.
Copy the file
Run Mathematica.
Load NCGBTestCreate.m
Run
Exit Mma
Here the arguments Integer must be a non-negative integer, satisfying Integer1 ≤ Integer2 ≤ 85 This creates a set of files of answers obtained from NCProcess labelled
where N is an integer between Integer1 and Integer2. They are stored in the directory TestResults.
If you want to test a program besides NCProcess you can use
Label whichever program you want to test with FunctionName. For example, the name NCMakeGBFunction has already been assigned to NCMakeGB, so running
will create files for testing NCMakeGB.
Comparing to Reference Code
While in the testing directory
Run
NCGBTestCompare[ Integer, Integer] (* for *.tex files only *)
or to test programs other than NCProcess
NCGBTestCompare[ Integer, Integer, suffix]
(* for example, suffix is ”GB” only — remember the quotes *)
Go get a cup of coffee. When you come back look at the resulting screen output.
You should get
The command NCGBTestCompare has created answer files and compared them to the old reference files.
Creating Reference Answers
Maybe you do not like the reference results we provided to you (some people are not very grateful). To create a new set of reference outputs stay in the Testing directory. Make sure you have lots of test problem files in the TestProblems directory; say c01.data.m thru c78.data.m.
Edit 2 lines in the file
to put in the path to the version of NCGB that you want as a reference.
Copy the file
Run Mathematica
Load NCGBTestCreate.m
Run
This is beginning to look familiar. From here on proceed as you did in creating test files. This creates a collection of files in the directory TestReference.