unitizeR - Miscellanea

Default Mode is to Store Tests in rds Files

unitizer stores unit tests and their results. By default, it stores them in rds files in your filesystem. You will be prompted before a file is saved to your filesystem.

The rds file is placed in a directory with the same name as your test file, but with “unitizer” appended. For example, if your tests are in “my_file_name.R”, then unitizer will create a folder called “my_file_name.unitizer/” and put an rds file in it.

See ?get_unitizer for potential alternatives to saving to your file system.

File Space Considerations

If your tests produce massive objects, the unitizer rds file will be massive. Try designing your tests so they will produce the smallest representative data structures needed for your tests to be useful.

Additionally, note that the rds files are binary, which needs to be accounted for when using them in version controlled projects.

Backup Your unitizer Stores

unitizer does not backup the rds beyond the single copy in the aforementioned folder. Unit tests are valuable, and without the rds file unitizer tests become a lot less useful. To the extent you backup your R test files, you should also backup the corresponding “.unitizer/” folder. You could lose / corrupt your unitizer store in many ways. Some non-exhaustive examples:

• Standard file system SNAFU
• Careless updates to existing unitizer
• unitizer developer accidentally introduces a bug that destroys your unitizer

Backup your unitizer stores!

Alternate Store Locations

unitize stores and loads unitizers using the set_unitizer and get_unitizer S3 generics . This means you can implement your own S3 methods for those generics to store the unitizer object off-filesystem (e.g. MySQL databse, etc). See ?get_unitizer for more details, though note this feature is untested.

If you only wish to save your unitizer to a different location in your filesystem than the default, you do not need to resort to these methods as you can provide the target directory with unitize(..., store.id=).

Committing Binary Files

The main issue with using unitizer with a version controlled package is that you have to decide whether you want to include the binary rds files in the version control history. Some options:

• Do not track the binary files at all (but they are valuable and now not backed up).
• Do not track the binary files at all, but implement a secondary back-up system (this sounds really annoying).
• Use a backed-up, non-file system store (see “Alternate Store Locations” above).
• Track the binary files, but manage how often they are committed.

We recommend splitting tests for different functionality into different files. This should mitigate the number of rds files that change with any given source code update, and is good practice anyway. Additionally, we typically only commit the rds files when a feature branch or issue resolution is fully complete.

Additionally a useful git shortcut to add to your .gitconfig file that mitigates how often you commit rds files is:

[alias]
        ad = !git add -u && git reset -- *.rds

This makes it easy to add all the files you are working on except for the rdses. Once you have stabilized a set of tests you can commit the rds.

All this aside, remember that the rdses are ultimately just as important as the test files, and you should commit them occasionally to ensure you do not use valuable test information.

Collaborating with Unitizer

If you merge in a pull request from a third party you do not fully trust, we recommend that you do not accept any commits to the rdses. You can accept and review changes to test expressions, and then unitize against your existing rdses and review the corresponding values.

review

review allows you to review all tests in a unitizer rds with the option of dropping tests from it. See ?review.

editCalls

Warning: this is experimental; make sure your test store is backed up before you use it.

editCalls allows you to modify the calls calls stored in a unitizer. This is useful when you decide to change the call (e.g. a function name), but otherwise leave the behavior of the call unchanged. You can then upate your test script and the renamed calls will be matched against the correct values in the unitizer store. Without this you would have to re-review and re-store every test since unitizer identifies tests by the deparsed call.

split

There is currently no direct way to split a unitizer into pieces (see issue #44), but the current work around is to:

1. Copy the test file and the corresponding unitizer to a new location.
2. Edit the original test file to remove the tests we want to split off.
3. Run unitizer and agree to drop all removed tests (hint: this is a good time to use YY).
4. Edit the new test file and remove the tests that are still in the old test file.
5. Run unitizer and agree to drop all removed tests.

The net result will be two new unitizer, each with a portion of the tests from the original unitizer. Clearly less than ideal, but will work in a pinch.

After Running unitizer Output No Longer Shows on Screen

unitizer sinks stdout and stderr during test evaluation, so it is possible that in some corner cases unitizer exits without releasing sinks. We have put substantial effort in trying to avoid this eventuality, but should it occur, here are some things you can do:

• Run: while(sink.number()) sink() and sink(type="message") to reset the output stream sinks.
• Or, restart the R session (type q() followed by ENTER, then “y” or “n” (without quotes) depending on whether you want to save your workspace or not).

Either way, please contact the maintainer as this should not happen.

unitizer Freezes and Pops up “Selection:”

This is almost certainly a result of an R crash. Unfortunately the normal mechanisms to restore stderr don’t seem to work completely with full R crashes, so when you see things like:

+------------------------------------------------------------------------------+
| unitizer for: tests/unitizer/alike.R                                         |
+------------------------------------------------------------------------------+

Running: alike(data.frame(a = integer(), b = factor()), data.frame(a = 1:3, Selection:

what you are not seeing is:

 *** caught segfault ***
address 0x7fdc20000010, cause 'memory not mapped'

Traceback:
 1: .Call(ALIKEC_alike, target, current, int.mode, int.tol, attr.mode)
 2: alike(data.frame(a = factor(), b = factor()), data.frame(a = 1:3,     b = letters[1:3]))

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace

The “Selection:” bit is prompting you to type 1-4 as per above. We will investigate to see if there is a way to address this problem, but the solution likely is not simple since the R crash circumvents the on.exit handlers used to reset the stream redirects. Also, note that in this case the crash is caused by alike, not unitizer (see below).

Running unitizer Crashes R

Every R crash we have discovered while using unitizer was eventually traced to a third party package. Some of the crashes were linked to issues attaching/detaching packages. If you think you might be having an issue with this you can always turn this feature off via the state parameter (not the feature is off by default).

Different Outcomes in Interactive vs. Non Interactive

Watch out for functions that have default arguments of the type:

fun <- function(x, y=getOption('blahblah'))

as those options may be different depending on whether you are running whether you are running R interactively or not. One prime example is parse(..., keep.source = getOption("keep.source")).

Running unitize Within Error Handling Blocks

Because unitize evaluates test expressions within a call to withCallingHandlers, there are some limitations on successfully running unitize inside your own error handling calls. In particular, unitize will not work properly if run inside a tryCatch or try statement. If test expressions throw conditions, the internal withCallingHandlers will automatically hand over control to your tryCatch/try statement without an opportunity to complete unitize computations. Unfortunately there does not seem to be a way around this since we have to use withCallingHandlers so that test statements after non-aborting conditions are run.

See this SO Q/A for more details on the problem.

Overridden Functions

In order to perpetuate the R console prompt illusion, unitizer needs to override some buit-in functionality, including:

• ls is replaced by a special version that can explore the unitizerItem environments
• quit and q are wrappers around the base functions that allow unitizer to quit gracefully
• traceback and .traceback are replaced to read the internally stored traces of the unitizer-handled errors in tests.
• History is replaced during unitizer prompt evaluations with a temporary version of the history file containing only commands evaluated at the unitizer prompt. The normal history file is restored on exit.