Guide for Writing Rbq-Lua Unit Tests

• Purpose of Unit Tests
• The Rbq-Lua API
• The Lua High-Level Libraries
• Location of Rbq-Lua Unit Tests in the Source Tree
• Two Types of Unit Tests: 'Immediate' and 'Tick'
• Running Multiple Unit Tests with the Test Driver
• An Example Unit Test
• Test Cases
• The TestUtil Lua Library
• Zero-based Indices and One-based Indices
• Creating a Known Testing Context
• enableLuaError()

Unit tests will not catch all bugs. Many bugs are exposed only through the combined action of multiple components, and must be found in other ways. Still, unit tests are a good baseline for maintaining program stability.

The core application code provides a number of operations that a user can perform, such as loading models into the program, retrieving data about those models, controlling the visual display, and so on. Such operations can be performed by clicking the menus, toolbars and typing commands into the graphical user interface. Operations like these can also be performed by executing scripting code that is contained in text files, where the scripts are written in the Lua scripting language.

In Lua scripts, the user can define global and local variables, expressions, if statements, for loops, and functions. The embedded Lua interpretor also provides a basic set of library functions to do string manipulation, mathematical calculation and file i/o. However, to interact with the Ribosome Builder application, a Lua script can only do so by calling a special set of functions that have been registered by the core application. These functions are all written in C and are defined in a single C++ file called CRbqLua.cpp. They all begin with the prefix 'rbq_'. Inside these functions, parameters are converted from the Lua scripting context into C numeric and string data types. Then, after the result of the function has been retrieved by calling C++ functions in the application core code, the return values are converted into Lua data types and passed back, via the script interpretor, to the scripting code.

HTML documentation for the Rbq-Lua API is automatically generated from comment header blocks that precede these functions. This documentation describes the purpose of each function, the number and type of parameters and return values, and sometimes additional notes and cross-referencing to related functions. The documentation organizes the functions into different categories. Occaisionally, a function will appear in more than one category because its purpose can not be fully localized to a single category.

The functions in the Lua Libraries also have HTML documentation that is automatically generated from comment header blocks that precede the functions. Some of the functions are simple wrappers around the rbq_XXX versions that are implemented in C, but many other functions implement higher-level operations in a variety of categories. These Lua library functions can also be used in the writing the Rbq-Lua unit tests to provide support operations such as opening documents, building paths, and unit-test specific operations, as provided by the _TestUtil.lua module, discussed in more detail below.

At some point, unit tests for the functions of the Lua Libraries should also be created.

Within this directory, there is a 'funcs' subdirectory. In the funcs subdirectory, the unit tests for each rbq_XXX function are located in subdirectories with the same name as the function. For an 'immediate' unit test of a function, a file called 'test.lua' is created within the function directory. For example, the 'immediate' unit test for the function 'rbq_InsPdbFile()' would have the full path 'bas/app/rbq/tst/rbqlua/funcs/rbqInsPdbFile/test.lua'.

This execution model is sufficient for testing some but not all of the functions in the Rbq-Lua API. There are functions which cannot be properly tested in such a 'single-pass' execution model. For example, some functions can initiate changes in the graphics display which are performed only through a sequence of queued events in the Qt windowing system. This requires that the testing script terminate execution and return to the caller in order for the desired behaviour to subsequently appear.

To test such delayed behaviour, a second type of unit test has been defined. This type is called 'tick', because it makes use of the 'tick' event of script objects to implement the unit test. The design of a 'tick' unit test is a little more complicated than that of an 'immediate' unit test. For example, the unit test for the function rbq_DrawOrthoText() is tickTest.lua. Because it defines script object events, it also consists of two additional files, tickTest.tick.lua and tickTest.draw.lua.

In addition, the driver script will create two files, a test result file and a test log file, that will receive data from each unit script as it is run. The result file simply holds the PASS/FAIL status of each test. The log file can hold more detailed information, depending upon what the implementer of a unit test script has decided to log.

In addition to running all unit tests, it is sometimes convienient to just run a subset of tests, such as the test you have just written. This can also be done from a menu item in the same submenu mentioned above. The additional requirement is that the desired subset of tests to be run must be specified in the file 'bas/app/rbq/tst/rbqlua/subset.testNames'.

The main section at the bottom includes three Lua Library modules that define some functions used by the script:

	load( "FileUtil.lua" );
	load( "PdbUtil.lua" );
	load( "TestUtil.lua" );

Then it defines a table that holds references to the 5 functions above, each of which tests a particular scenario involving the Rbq-Lua function:

	local tCases =
	{
		TestCase1,
		TestCase2,
		TestCase3,
		TestCase4,
		TestCase5,
	};

The next line calls a function in the TestUtil.lua module to do some setup work at the start of the unit test. This setup work currently just clears the 'failed' flag and logs a test start message to the test logfile, but it is also a placeholder for future intialization:

	testBegin( g_sTestName );

The next section is just a for loop to call each function to do a particular test case:

	local iCase;
	for iCase = 1, getn( tCases ) do
		local tCase = tCases[iCase];
		if not tCase() then
			do break; end
		end
	end

Finally, a cleanup function is called, which outputs the test PASS/FAIL status to the test result file:

	testEnd( g_sTestName );

The number of possible test cases and the code within a particular test case function will vary, depending upon the function being tested. However, some conventions currently exist:

• The beginning of the Test Case is logged, using testLog(), reporting the full name of the test case: 'begin rbq_InsPdbFile.TestCase1'.
• The end of the Test Case is logged, using testLog(), reporting the full name: 'end rbq_InsPdbFile.TestCase1'.
• When a test fails, the testFailed() function is called with the test case name and a description of what failed.
• The description for each testFailed() call should be unique, so that the failing line in the test case code can be located from the log message.
• When a test fails, 'nil' is returned to abort execution of the subsequent test cases in the file.

In a script, before calling a function in the TestUtil library, the library must be 'included' by with a 'load' statement:

	load( "TestUtil.lua" );

Such 'load' statements must also be done when using functions in any other application Lua Library module.

In the Lua language, indices are one-based. For example, the first element in a Lua table is accessed with the index '1'. The Lua built-in library functions such as strsub() also refer to the first character in a string as starting at the '1' position. In contrast, in the C and C++ code of the core application, all offsets begin at position '0'. As a result, there will be some mixing of 0-based and 1-based indices in the Rbq-Lua API code.

The rule for determining 1-based versus 0-based indices is as follows: when calling rbq_XXX functions, either directly or indirectly, 0-based parameters will be used. When calling built-in Lua functions, either directly or indirectly, 1-based parameters will be used. If there is any question as to which is the case, the documentation should clearly state whether the parameters and return values are 0-based or 1-based.

For example, unit tests that load and check the presence of a pdb model may first close any current document and create a new, empty document. A unit test that checks the result of a graphical function will first set the graphics window to be a specified size, so that the graphic snapshot will match a reference file. The helper script 'rlutSetGraphicWin.lua' can be used for setting the graphic window state.

The calling of 'lua_error()' is temporarily disabled through a call to the 'enableLuaError()' function during unit test cases that call rbq-lua functions with invalid parameters.